Kinect=002=skeleton tracking 2d by gary chang

KINECT Skeleton Tracking 2D

REVIEW

import SimpleOpenNI.*; SimpleOpenNI context; /////////////////////////////////////////////// void setup(){ context = new SimpleOpenNI (this); context.enableDepth(); context.enableUser();

Try to explain ? what does each lines mean

if(context.isInit() == false){ println("Maybe your Kinect is not connected"); exit(); return; } size(context.depthWidth(), context.depthHeight()); } /////////////////////////////////////////////// void draw(){ context.update(); image(context.userImage(),0,0); }

import SimpleOpenNI.*; SimpleOpenNI context;

Import library. Create an simpleOpenNI object.

/////////////////////////////////////////////// void setup(){ context = new SimpleOpenNI (this); context.enableDepth(); context.enableUser();

Initiate this simpleOpenNI object. Enable Depth calculation. Enable User(ID) calculation.

Detect Kinect’s connection

Size of the Window

Update the data Draw User’s Coloring Image

If you are IN A HURRY

Open the user example to get it.

But I will Try to Simplify the code

import SimpleOpenNI.*; SimpleOpenNI context; color[]

userClr = new color[]{ color(255,0,0), color(0,255,0), color(0,0,255), color(255,255,0), color(255,0,255), color(0,255,255) };

/////////////////////////////////////////////// void setup(){ context = new SimpleOpenNI (this); context.enableDepth(); context.enableUser(); if(context.isInit() == false){ println("Maybe your Kinect is not connected"); exit(); return; } size(context.depthWidth(), context.depthHeight()); } /////////////////////////////////////////////// void draw(){ context.update(); image(context.userImage(),0,0); }

If you try to compare the differences between the user example and our previous code.

Here is the first difference: Color for idetifying the users Which means each of the user identified by Kinect will has its own skeleton color.

import SimpleOpenNI.*; SimpleOpenNI context; color[]

userClr = new color[]{ color(255,0,0), color(0,255,0), color(0,0,255), color(255,255,0), color(255,0,255), color(0,255,255) };

But to , I donâ&#x20AC;&#x2122;t need it. (you can still keep it if you like).

import SimpleOpenNI.*; SimpleOpenNI context;

PVector com = new PVector(); PVector com2d = new PVector();

Second Difference: Center of Mass// Center of your figure

import SimpleOpenNI.*; SimpleOpenNI context;

PVector com = new PVector(); PVector com2d = new PVector(); /////////////////////////////////////////////// void setup(){ context = new SimpleOpenNI (this); context.enableDepth(); context.enableUser();

Center of Mass// Center of your figure If we want to keep it simple for the first time, we can still ignore them. (or talk about it later).

ARE YOU KIDDING ME?

Here comes the REAL stuffs.

///////////////////////////////////////////////

void draw(){ context.update(); image(context.userImage(),0,0);

int [] userList = context.getUsers();

Detect the users.

for(int i =0; i<userList.length; i++){

For every users, they will â&#x20AC;Ś

if(context.isTrackingSkeleton(userList[i])){ If Kinect stroke(255,0,0); drawSkeleton(userList[i]); Draw the skeleton } } }

have tracked someoneâ&#x20AC;Ś

But we donâ&#x20AC;&#x2122;t have drawSkeleton(userList[i]) function yet!

HEAD

RIGHT_SHOULDER

LEFT_SHOULDER NECK

RIGHT_ELBOW

LEFT_ELBOW TORSO RIGHT_HIP

LEFT_HIP

RIGHT_HAND

LEFT_HAND

RIGHT_KNEE

RIGHT_FOOT

LEFT_KNEE

LEFT_FOOT

Here you are

(simply copy from example)

void drawSkeleton(int userId) { // to get the 3d joint data /* PVector jointPos = new PVector(); context.getJointPositionSkeleton(userId,SimpleOpenNI.SKEL_NECK,jointPos); println(jointPos); */ context.drawLimb(userId, SimpleOpenNI.SKEL_HEAD, SimpleOpenNI.SKEL_NECK); context.drawLimb(userId, SimpleOpenNI.SKEL_NECK, SimpleOpenNI.SKEL_LEFT_SHOULDER); context.drawLimb(userId, SimpleOpenNI.SKEL_LEFT_SHOULDER, SimpleOpenNI.SKEL_LEFT_ELBOW); context.drawLimb(userId, SimpleOpenNI.SKEL_LEFT_ELBOW, SimpleOpenNI.SKEL_LEFT_HAND); context.drawLimb(userId, SimpleOpenNI.SKEL_NECK, SimpleOpenNI.SKEL_RIGHT_SHOULDER); context.drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_SHOULDER, SimpleOpenNI.SKEL_RIGHT_ELBOW); context.drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_ELBOW, SimpleOpenNI.SKEL_RIGHT_HAND); context.drawLimb(userId, SimpleOpenNI.SKEL_LEFT_SHOULDER, SimpleOpenNI.SKEL_TORSO); context.drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_SHOULDER, SimpleOpenNI.SKEL_TORSO); context.drawLimb(userId, SimpleOpenNI.SKEL_TORSO, SimpleOpenNI.SKEL_LEFT_HIP); context.drawLimb(userId, SimpleOpenNI.SKEL_LEFT_HIP, SimpleOpenNI.SKEL_LEFT_KNEE); context.drawLimb(userId, SimpleOpenNI.SKEL_LEFT_KNEE, SimpleOpenNI.SKEL_LEFT_FOOT); context.drawLimb(userId, SimpleOpenNI.SKEL_TORSO, SimpleOpenNI.SKEL_RIGHT_HIP); context.drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_HIP, SimpleOpenNI.SKEL_RIGHT_KNEE); context.drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_KNEE, SimpleOpenNI.SKEL_RIGHT_FOOT); }

Let’s take a look!

Function context.drawLimb(userId, SimpleOpenNI.SKEL_NECK, SimpleOpenNI.SKEL_RIGHT_SHOULDER); context.drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_SHOULDER, SimpleOpenNI.SKEL_RIGHT_ELBOW); context.drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_ELBOW, SimpleOpenNI.SKEL_RIGHT_HAND);

Object

Feed in Data 1: UserID

Feed in Data 2: Draw Something

Donâ&#x20AC;&#x2122;t be afraid of Copy&Paste, Unless you know how to Change. (Coding is not an memorizing game)

The Last piece of your Skeleton puzzle.

Lost&Found Users

(simply copy from example)

void onNewUser(SimpleOpenNI curContext, int userId) { println("onNewUser - userId: " + userId); println("\tstart tracking skeleton"); context.startTrackingSkeleton(userId); } void onLostUser(SimpleOpenNI curContext, int userId) { println("onLostUser - userId: " + userId); } void onVisibleUser(SimpleOpenNI curContext, int userId) { //println("onVisibleUser - userId: " + userId); }

import SimpleOpenNI.*; SimpleOpenNI context; /////////////////////////////////////////////// void setup(){ context = new SimpleOpenNI (this); context.enableDepth(); context.enableUser();

if(context.isInit() == false){ println("Maybe your Kinect is not connected"); exit(); return; }

context.drawLimb(userId, SimpleOpenNI.SKEL_NECK, SimpleOpenNI.SKEL_RIGHT_SHOULDER); context.drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_SHOULDER, SimpleOpenNI.SKEL_RIGHT_ELBOW); context.drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_ELBOW, SimpleOpenNI.SKEL_RIGHT_HAND); context.drawLimb(userId, SimpleOpenNI.SKEL_LEFT_SHOULDER, SimpleOpenNI.SKEL_TORSO); context.drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_SHOULDER, SimpleOpenNI.SKEL_TORSO); context.drawLimb(userId, SimpleOpenNI.SKEL_TORSO, SimpleOpenNI.SKEL_LEFT_HIP); context.drawLimb(userId, SimpleOpenNI.SKEL_LEFT_HIP, SimpleOpenNI.SKEL_LEFT_KNEE); context.drawLimb(userId, SimpleOpenNI.SKEL_LEFT_KNEE, SimpleOpenNI.SKEL_LEFT_FOOT);

size(context.depthWidth(), context.depthHeight()); } /////////////////////////////////////////////// void draw(){ context.update(); image(context.userImage(),0,0); int [] userList = context.getUsers(); for(int i =0; i<userList.length; i++){ if(context.isTrackingSkeleton(userList[i])){ stroke(255,0,0); drawSkeleton(userList[i]); } } }

context.drawLimb(userId, SimpleOpenNI.SKEL_TORSO, SimpleOpenNI.SKEL_RIGHT_HIP); context.drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_HIP, SimpleOpenNI.SKEL_RIGHT_KNEE); context.drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_KNEE, SimpleOpenNI.SKEL_RIGHT_FOOT); } // SimpleOpenNI user events/////////////////////////////////////////////////////////////////////////////////////// void onNewUser(SimpleOpenNI curContext, int userId) { println("onNewUser - userId: " + userId); println("\tstart tracking skeleton"); context.startTrackingSkeleton(userId); } void onLostUser(SimpleOpenNI curContext, int userId) { println("onLostUser - userId: " + userId); } void onVisibleUser(SimpleOpenNI curContext, int userId) { //println("onVisibleUser - userId: " + userId); }

Free As A Bird

THE color: (255,0,0)

I want my Head! Letâ&#x20AC;&#x2122;s create a function!

void drawHead(int userId){ PVector jointPosH = new PVector(); //Create a Vector (to capture the Head joint). context.getJointPositionSkeleton(userId, SimpleOpenNI.SKEL_HEAD, jointPosH); //Get the Head Joint in 3D! PVector jointPosH_Proj = new PVector(); //Create a Vector (for projecting on 2D). context.convertRealWorldToProjective(jointPosH, jointPosH_Proj); //Convert 3D Head Joint Vector to 2D screen (for projecting on 2D).

fill(0,255,0); stroke(255,0,0); ellipse(jointPosH_Proj.x, jointPosH_Proj.y, 50, 50); // Extract the x and y position from the Projection Head Joint to draw a circle.

}

Put it in the CODE

import SimpleOpenNI.*; SimpleOpenNI context; /////////////////////////////////////////////// void setup(){ context = new SimpleOpenNI (this); context.enableDepth(); context.enableUser();

context.drawLimb(userId, SimpleOpenNI.SKEL_TORSO, SimpleOpenNI.SKEL_LEFT_HIP); context.drawLimb(userId, SimpleOpenNI.SKEL_LEFT_HIP, SimpleOpenNI.SKEL_LEFT_KNEE); context.drawLimb(userId, SimpleOpenNI.SKEL_LEFT_KNEE, SimpleOpenNI.SKEL_LEFT_FOOT); context.drawLimb(userId, SimpleOpenNI.SKEL_TORSO, SimpleOpenNI.SKEL_RIGHT_HIP); context.drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_HIP, SimpleOpenNI.SKEL_RIGHT_KNEE); context.drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_KNEE, SimpleOpenNI.SKEL_RIGHT_FOOT); } // SimpleOpenNI user events/////////////////////////////////////////////////////////////////////////////////////// void onNewUser(SimpleOpenNI curContext, int userId) { println("onNewUser - userId: " + userId); println("\tstart tracking skeleton"); context.startTrackingSkeleton(userId); } void onLostUser(SimpleOpenNI curContext, int userId) { println("onLostUser - userId: " + userId); } void onVisibleUser(SimpleOpenNI curContext, int userId) { //println("onVisibleUser - userId: " + userId); }

void drawHead(int userId){

int [] userList = context.getUsers();

PVector jointPosH = new PVector(); //Create a Vector (to capture the Head joint).

for(int i =0; i<userList.length; i++){ context.getJointPositionSkeleton(userId, SimpleOpenNI.SKEL_HEAD, jointPosH); //Get the Head Joint in 3D!

if(context.isTrackingSkeleton(userList[i])){ stroke(255,0,0); drawSkeleton(userList[i]); drawHead(userList[i]); }

PVector jointPosH_Proj = new PVector(); //Create a Vector (for projecting on 2D).

context.convertRealWorldToProjective(jointPosH, jointPosH_Proj); //Convert 3D Head Joint Vector to 2D screen (for projecting on 2D).

}

fill(0,255,0); stroke(255,0,0); ellipse(jointPosH_Proj.x, jointPosH_Proj.y, 80, 80); // Extract the x and y position from the Projection Head Joint to draw a circle.

} }

Letâ&#x20AC;&#x2122;s reduce Some stuffs

import SimpleOpenNI.*; SimpleOpenNI context; /////////////////////////////////////////////// void setup(){ context = new SimpleOpenNI (this); context.enableDepth(); context.enableUser();

void drawHead(int userId){ PVector jointPosH = new PVector(); //Create a Vector (to capture the Head joint).

int [] userList = context.getUsers();

context.getJointPositionSkeleton(userId, SimpleOpenNI.SKEL_HEAD, jointPosH); //Get the Head Joint in 3D!

for(int i =0; i<userList.length; i++){ if(context.isTrackingSkeleton(userList[i])){ stroke(255,0,0); drawSkeleton(userList[i]); drawHead(userList[i]); }

PVector jointPosH_Proj = new PVector(); //Create a Vector (for projecting on 2D).

context.convertRealWorldToProjective(jointPosH, jointPosH_Proj); //Convert 3D Head Joint Vector to 2D screen (for projecting on 2D).

}

fill(0,255,0); stroke(255,0,0); ellipse(jointPosH_Proj.x, jointPosH_Proj.y, 80, 80); // Extract the x and y position from the Projection Head Joint to draw a circle.

} }

In this Virtual World NO One is FAT!

Change the Color You Like

import SimpleOpenNI.*; SimpleOpenNI context; /////////////////////////////////////////////// void setup(){ context = new SimpleOpenNI (this); context.enableDepth(); context.enableUser();

if(context.isInit() == false){ println("Maybe your Kinect is not connected"); exit(); return; } size(context.depthWidth(), context.depthHeight()); } /////////////////////////////////////////////// void draw(){ background(0); context.update(); //image(context.userImage(),0,0);

void drawHead(int userId){ PVector jointPosH = new PVector(); //Create a Vector (to capture the Head joint).

int [] userList = context.getUsers();

context.getJointPositionSkeleton(userId, SimpleOpenNI.SKEL_HEAD, jointPosH); //Get the Head Joint in 3D!

for(int i =0; i<userList.length; i++){ if(context.isTrackingSkeleton(userList[i])){ strokeWeight(2); stroke(0,255,255); drawSkeleton(userList[i]); drawHead(userList[i]); }

PVector jointPosH_Proj = new PVector(); //Create a Vector (for projecting on 2D).

context.convertRealWorldToProjective(jointPosH, jointPosH_Proj); //Convert 3D Head Joint Vector to 2D screen (for projecting on 2D). fill(0,255,255); stroke(0,255,255.80); ellipse(jointPosH_Proj.x, jointPosH_Proj.y, 50, 50); // Extract the x and y position from the Projection Head Joint to draw a circle.

} } }

Duchamp Effect Nude Descending a Staircase No. 2 || Marcel Duchamp'

import SimpleOpenNI.*; SimpleOpenNI context; /////////////////////////////////////////////// void setup(){ context = new SimpleOpenNI (this); context.enableDepth(); context.enableUser();

if(context.isInit() == false){ println("Maybe your Kinect is not connected"); exit(); return; } size(context.depthWidth(), context.depthHeight()); background(0); } /////////////////////////////////////////////// void draw(){ context.update(); //image(context.userImage(),0,0);

void drawHead(int userId){ PVector jointPosH = new PVector(); //Create a Vector (to capture the Head joint).

int [] userList = context.getUsers(); context.getJointPositionSkeleton(userId, SimpleOpenNI.SKEL_HEAD, jointPosH); //Get the Head Joint in 3D!

for(int i =0; i<userList.length; i++){

PVector jointPosH_Proj = new PVector(); //Create a Vector (for projecting on 2D).

if(context.isTrackingSkeleton(userList[i])){ strokeWeight(2); stroke(0,255,255,80); drawSkeleton(userList[i]); drawHead(userList[i]); }

context.convertRealWorldToProjective(jointPosH, jointPosH_Proj); //Convert 3D Head Joint Vector to 2D screen (for projecting on 2D). fill(0,255,255,80); stroke(0,255,255.80); ellipse(jointPosH_Proj.x, jointPosH_Proj.y, 50, 50); // Extract the x and y position from the Projection Head Joint to draw a circle.

} }

}

[ If-else ] Again

import SimpleOpenNI.*; SimpleOpenNI context; /////////////////////////////////////////////// void setup(){ context = new SimpleOpenNI (this); context.enableDepth(); context.enableUser(); if(context.isInit() == false){ println("Maybe your Kinect is not connected"); exit(); return; } size(context.depthWidth(), context.depthHeight()); } /////////////////////////////////////////////// void draw(){ background(0); context.update(); //image(context.userImage(),0,0); int [] userList = context.getUsers();

for(int i =0; i<userList.length; i++){ if(context.isTrackingSkeleton(userList[i])){

PVector jointPosH = new PVector(); //Create a Vector (to capture the Head joint). context.getJointPositionSkeleton(userId, SimpleOpenNI.SKEL_HEAD, jointPosH); //Get the Head Joint in 3D!

strokeWeight(2); stroke(0,255,255,80); drawSkeleton(userList[i]); drawHead(userList[i]);

PVector myHead = new PVector(); context.getJointPositionSkeleton(userList[i], SimpleOpenNI.SKEL_HEAD, myHead); PVector myHead_Proj = new PVector(); context.convertRealWorldToProjective(myHead, myHead_Proj); if(myHead_Proj.x> width/2){ background(255,0,255); } else{ background(255,255,0); }

} } }

PVector jointPosH_Proj = new PVector(); //Create a Vector (for projecting on 2D). context.convertRealWorldToProjective(jointPosH, jointPosH_Proj); //Convert 3D Head Joint Vector to 2D screen (for projecting on 2D). fill(0,255,255,80); stroke(0,255,255,80); ellipse(jointPosH_Proj.x, jointPosH_Proj.y, 50, 50); // Extract the x and y position from the Projection Head Joint to draw a circle. }

else

if(myHead_Proj.x > width/2)

But, Dude, Where is my BODY? (you should know)

for(int i =0; i<userList.length; i++){ if(context.isTrackingSkeleton(userList[i])){

PVector jointPosH = new PVector(); //Create a Vector (to capture the Head joint). context.getJointPositionSkeleton(userId, SimpleOpenNI.SKEL_HEAD, jointPosH); //Get the Head Joint in 3D!

} } }

fill(0,255,255); stroke(0,255,255); ellipse(jointPosH_Proj.x, jointPosH_Proj.y, 50, 50); // Extract the x and y position from the Projection Head Joint to draw a circle. }

else

if(myHead_Proj.x > width/2)

Here you go

[ If-else ] Change Head to Hand

import SimpleOpenNI.*; SimpleOpenNI context; /////////////////////////////////////////////// void setup(){ context = new SimpleOpenNI (this); context.enableDepth(); context.enableUser();

if(context.isInit() == false){ println("Maybe your Kinect is not connected"); exit(); return; } size(context.depthWidth(), context.depthHeight());

} ///////////////////////////////////////////////

context.drawLimb(userId, SimpleOpenNI.SKEL_LEFT_SHOULDER, SimpleOpenNI.SKEL_TORSO); context.drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_SHOULDER, SimpleOpenNI.SKEL_TORSO);

void draw(){

background(0); context.update(); //image(context.userImage(),0,0); int [] userList = context.getUsers();

for(int i =0; i<userList.length; i++){ if(context.isTrackingSkeleton(userList[i])){ PVector myRHand = new PVector(); context.getJointPositionSkeleton(userList[i], SimpleOpenNI.SKEL_RIGHT_HAND, myRHand); PVector myRHand_Proj = new PVector(); context.convertRealWorldToProjective(myRHand, myRHand_Proj); if(myRHand_Proj.x> width/2){ background(255,0,255); } else{ background(255,255,0); }

// SimpleOpenNI user events/////////////////////////////////////////////////////////////////////////////////////// void onNewUser(SimpleOpenNI curContext, int userId) { println("onNewUser - userId: " + userId); println("\tstart tracking skeleton"); context.startTrackingSkeleton(userId); } void onLostUser(SimpleOpenNI curContext, int userId) { println("onLostUser - userId: " + userId); } void onVisibleUser(SimpleOpenNI curContext, int userId) { //println("onVisibleUser - userId: " + userId); } void drawHead(int userId){ PVector jointPosH = new PVector(); //Create a Vector (to capture the Head joint). context.getJointPositionSkeleton(userId, SimpleOpenNI.SKEL_HEAD, jointPosH); //Get the Head Joint in 3D! PVector jointPosH_Proj = new PVector(); //Create a Vector (for projecting on 2D).

strokeWeight(2); stroke(0,255,255); drawSkeleton(userList[i]); drawHead(userList[i]); stroke(0,255,255); strokeWeight(2); fill(0,255,0); ellipse(myRHand_Proj.x,myRHand_Proj.y,50,50); stroke(0,255,255); strokeWeight(2); fill(255,0,255); ellipse(myRHand_Proj.x,myRHand_Proj.y,25,25);

} } }

context.convertRealWorldToProjective(jointPosH, jointPosH_Proj); //Convert 3D Head Joint Vector to 2D screen (for projecting on 2D).

fill(0,255,255); stroke(0,255,255); ellipse(jointPosH_Proj.x, jointPosH_Proj.y, 50, 50); // Extract the x and y position from the Projection Head Joint to draw a circle. }

else

if(myHead_Proj.x > width/2)

GIVE ME 2 HANDS [ no If-else ]

import SimpleOpenNI.*; SimpleOpenNI context; /////////////////////////////////////////////// void setup(){ context = new SimpleOpenNI (this); context.enableDepth(); context.enableUser();

if(context.isInit() == false){ println("Maybe your Kinect is not connected"); exit(); return; } size(context.depthWidth(), context.depthHeight()); } ///////////////////////////////////////////////

context.drawLimb(userId, SimpleOpenNI.SKEL_NECK, SimpleOpenNI.SKEL_LEFT_SHOULDER); context.drawLimb(userId, SimpleOpenNI.SKEL_LEFT_SHOULDER, SimpleOpenNI.SKEL_LEFT_ELBOW); context.drawLimb(userId, SimpleOpenNI.SKEL_LEFT_ELBOW, SimpleOpenNI.SKEL_LEFT_HAND); context.drawLimb(userId, SimpleOpenNI.SKEL_NECK, SimpleOpenNI.SKEL_RIGHT_SHOULDER); context.drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_SHOULDER, SimpleOpenNI.SKEL_RIGHT_ELBOW); context.drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_ELBOW, SimpleOpenNI.SKEL_RIGHT_HAND); context.drawLimb(userId, SimpleOpenNI.SKEL_LEFT_SHOULDER, SimpleOpenNI.SKEL_TORSO); context.drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_SHOULDER, SimpleOpenNI.SKEL_TORSO);

void draw(){

background(0); context.update(); //image(context.userImage(),0,0); int [] userList = context.getUsers();

for(int i =0; i<userList.length; i++){ if(context.isTrackingSkeleton(userList[i])){ PVector myRHand = new PVector(); context.getJointPositionSkeleton(userList[i], SimpleOpenNI.SKEL_RIGHT_HAND, myRHand); PVector myRHand_Proj = new PVector(); context.convertRealWorldToProjective(myRHand, myRHand_Proj); PVector myLHand = new PVector(); context.getJointPositionSkeleton(userList[i], SimpleOpenNI.SKEL_LEFT_HAND, myLHand); PVector myLHand_Proj = new PVector(); context.convertRealWorldToProjective(myLHand, myLHand_Proj);

strokeWeight(2); stroke(0,255,255); drawSkeleton(userList[i]); drawHead(userList[i]); stroke(0,255,255); strokeWeight(2); fill(0,255,0); ellipse(myRHand_Proj.x,myRHand_Proj.y,50,50); ellipse(myLHand_Proj.x,myRHand_Proj.y,50,50); stroke(0,255,255); strokeWeight(2); fill(255,0,255); ellipse(myRHand_Proj.x,myRHand_Proj.y,25,25); ellipse(myLHand_Proj.x,myRHand_Proj.y,25,25);

} } }

fill(0,255,255); stroke(0,255,255); ellipse(jointPosH_Proj.x, jointPosH_Proj.y, 50, 50); // Extract the x and y position from the Projection Head Joint to draw a circle. }

We got hands!

Please save this file.

Make V-buttons

I will only keep the â&#x20AC;&#x153;void draw()â&#x20AC;? part for now, since there will not be too many changes in the rest.

void draw(){

strokeWeight(2); stroke(0,255,255); drawSkeleton(userList[i]); drawHead(userList[i]);

We need pre-set buttons

int [] userList = context.getUsers(); for(int i =0; i<userList.length; i++){ if(context.isTrackingSkeleton(userList[i])){ PVector myRHand = new PVector(); context.getJointPositionSkeleton(userList[i], SimpleOpenNI.SKEL_RIGHT_HAND, myRHand); PVector myRHand_Proj = new PVector(); context.convertRealWorldToProjective(myRHand, myRHand_Proj); PVector myLHand = new PVector(); context.getJointPositionSkeleton(userList[i], SimpleOpenNI.SKEL_LEFT_HAND, myLHand); PVector myLHand_Proj = new PVector(); context.convertRealWorldToProjective(myLHand, myLHand_Proj);

stroke(0,255,255); strokeWeight(2); fill(0,255,0); ellipse(myRHand_Proj.x,myRHand_Proj.y,50,50); ellipse(myLHand_Proj.x,myRHand_Proj.y,50,50); stroke(0,255,255); strokeWeight(2); fill(255,0,255); ellipse(myRHand_Proj.x,myRHand_Proj.y,25,25); ellipse(myLHand_Proj.x,myRHand_Proj.y,25,25);

} }

}

Keep them transparent, as a default mode

If you move your hand to “?AREA”, it will trigger the buttons.

void draw(){

strokeWeight(2); stroke(0,255,255); drawSkeleton(userList[i]); drawHead(userList[i]);

background(0); context.update(); //Right button noStroke(); fill(255,0,255,80); ellipse(3*width/4,height/2,100,100); stroke(255,0,255,80); noFill(); ellipse(3*width/4,height/2,120,120); //Left button noStroke(); fill(255,255,0,80); ellipse(width/4,height/2,100,100); stroke(255,255,0,80); noFill(); ellipse(width/4,height/2,120,120); int [] userList = context.getUsers(); for(int i =0; i<userList.length; i++){ if(context.isTrackingSkeleton(userList[i])){ PVector myRHand = new PVector(); context.getJointPositionSkeleton(userList[i], SimpleOpenNI.SKEL_RIGHT_HAND, myRHand); PVector myRHand_Proj = new PVector(); context.convertRealWorldToProjective(myRHand, myRHand_Proj); PVector myLHand = new PVector(); context.getJointPositionSkeleton(userList[i], SimpleOpenNI.SKEL_LEFT_HAND, myLHand); PVector myLHand_Proj = new PVector(); context.convertRealWorldToProjective(myLHand, myLHand_Proj);

//R_Trigger if(myRHand_Proj.x<3*width/4+25 && myRHand_Proj.x>3*width/425 && myRHand_Proj.y<height/2+25 && myRHand_Proj.y>height/225){ noStroke(); fill(255,0,255); ellipse(3*width/4,height/2,100,100); stroke(255,0,255); noFill(); ellipse(3*width/4,height/2,120,120); } //L_Trigger if(myLHand_Proj.x<width/4+25 && myLHand_Proj.x>width/4-25 && myLHand_Proj.y<height/2+25 && myLHand_Proj.y>height/2-25){ noStroke(); fill(255,255,0); ellipse(width/4,height/2,100,100); stroke(255,255,0); noFill(); ellipse(width/4,height/2,120,120); } } }

}

Letâ&#x20AC;&#x2122;s make it clear.

//R_Trigger if(myRHand_Proj.x<3*width/4+25 && myRHand_Proj.x>3*width/4-25 && myRHand_Proj.y<height/2+25 && myRHand_Proj.y>height/2-25){ noStroke(); fill(255,0,255); ellipse(3*width/4,height/2,100,100); stroke(255,0,255); noFill(); ellipse(3*width/4,height/2,120,120); } //L_Trigger if(myLHand_Proj.x<width/4+25 && myLHand_Proj.x>width/4-25 && myLHand_Proj.y<height/2+25 && myLHand_Proj.y>height/2-25){ noStroke(); fill(255,255,0); ellipse(width/4,height/2,100,100); stroke(255,255,0); noFill(); ellipse(width/4,height/2,120,120); }

Try to explain, please.

(width/4, height/2)

else

if(myHead_Proj.x > width/2)

THINK: if I want not only Left hand for Left button Right hand for Right button

//R_Trigger if(myRHand_Proj.x<3*width/4+25 && myRHand_Proj.x>3*width/4-25 && myRHand_Proj.y<height/2+25 && myRHand_Proj.y>height/2-25 || myLHand_Proj.x<3*width/4+25 && myLHand_Proj.x>3*width/4-25 && myLHand_Proj.y<height/2+25 && myLHand_Proj.y>height/2-25) ){ noStroke(); fill(255,0,255); ellipse(3*width/4,height/2,100,100); stroke(255,0,255); noFill(); ellipse(3*width/4,height/2,120,120); } //L_Trigger if(myLHand_Proj.x<width/4+25 && myLHand_Proj.x>width/4-25 && myLHand_Proj.y<height/2+25 && myLHand_Proj.y>height/2-25 || myRHand_Proj.x<width/4+25 && myRHand_Proj.x>width/4-25 && myRHand_Proj.y<height/2+25 && myRHand_Proj.y>height/2-25 ){ noStroke(); fill(255,255,0); ellipse(width/4,height/2,100,100); stroke(255,255,0); noFill(); ellipse(width/4,height/2,120,120); }

Put “OR” in the code

You are the dancing queen

Please save another file.

Letâ&#x20AC;&#x2122;s make Some Noise!!!

Try to combine an example from “minim” library

Minim has been embedded in Processing.

Yours might look different than this one, but sure you can still find this â&#x20AC;&#x153;TriggerASampleâ&#x20AC;? file.

import ddf.minim.*; Minim minim; AudioSample kick; AudioSample snare; void setup() { size(512, 200, P3D); minim = new Minim(this); // load BD.wav from the data folder kick = minim.loadSample( "BD.mp3", // filename 512 // buffer size ); // if a file doesn't exist, loadSample will return null if ( kick == null ) println("Didn't get kick!"); // load SD.wav from the data folder snare = minim.loadSample("SD.wav", 512); if ( snare == null ) println("Didn't get snare!");

void draw() { background(0); stroke(255); // use the mix buffer to draw the waveforms. for (int i = 0; i < kick.bufferSize() - 1; i++) { float x1 = map(i, 0, kick.bufferSize(), 0, width); float x2 = map(i+1, 0, kick.bufferSize(), 0, width); line(x1, 50 - kick.mix.get(i)*50, x2, 50 kick.mix.get(i+1)*50); line(x1, 150 - snare.mix.get(i)*50, x2, 150 snare.mix.get(i+1)*50); } } void keyPressed() { if ( key == 's' ) snare.trigger(); if ( key == 'k' ) kick.trigger(); }

}

I simplify the code. The color parts are what we need. You can run the example, and press the S or K key on your keyboard.

I hope you should know How to do it!

import SimpleOpenNI.*; SimpleOpenNI context;

import ddf.minim.*; Minim minim; AudioSample kick; AudioSample snare;

void setup(){ context = new SimpleOpenNI (this); context.enableDepth(); context.enableUser();

minim = new Minim(this); kick = minim.loadSample( "BD.mp3", 512 ); // if a file doesn't exist, loadSample will return null if ( kick == null ) println("Didn't get kick!"); // load SD.wav from the data folder snare = minim.loadSample("SD.wav", 512); if ( snare == null ) println("Didn't get snare!"); if(context.isInit() == false){ println("Maybe your Kinect is not connected"); exit(); return; } size(context.depthWidth(), context.depthHeight());

}

For *import libraries, *declare *void setup()

For *void draw() All you need to do is put the trigger in the button

snare.trigger(); ){ noStroke(); fill(255,0,255); ellipse(3*width/4,height/2,100,100); stroke(255,0,255); noFill(); ellipse(3*width/4,height/2,120,120); } //L_Trigger if(myLHand_Proj.x<width/4+25 && myLHand_Proj.x>width/4-25 && myLHand_Proj.y<height/2+25 && myLHand_Proj.y>height/2-25 || myRHand_Proj.x<width/4+25 && myRHand_Proj.x>width/4-25 && myRHand_Proj.y<height/2+25 && myRHand_Proj.y>height/2-25 ){

kick.trigger(); noStroke(); fill(255,255,0); ellipse(width/4,height/2,100,100); stroke(255,255,0); noFill(); ellipse(width/4,height/2,120,120); }

I didn’t put the whole code, it’s too much in one slide.

I suppose you have saved your file all the time (I save it as â&#x20AC;&#x153;skeleton_Button_Soundâ&#x20AC;?)

Last thing here you have to do: Get the sound!!!

This is the path C:\Users\archgary\Desktop\processing-2.0.3-windows64\processing2.0.3\modes\java\libraries\minim\examples\TriggerASample ***Everyone might have different path, but just try to find your minim library folder, and inside the example, there should be the one.

Copy this “data” folder and put it in your “SAVED” file folder

Don…

Ts…

Now you are the drummer

Letâ&#x20AC;&#x2122;s play a bit pattern

Open the file With 2 hands

First, get the girds

import SimpleOpenNI.*; SimpleOpenNI context; ArrayList grid; import ddf.minim.*; Minim minim; AudioSample kick; AudioSample snare; void setup(){ context = new SimpleOpenNI (this); context.enableDepth(); context.enableUser(); grid = new ArrayList(); if(context.isInit() == false){ println("Maybe your Kinect is not connected"); exit(); return; } size(context.depthWidth(), context.depthHeight()); for(int i =0; i<= width; i+=48){ for(int j=0; j<= height; j+=48){ PVector loc = new PVector(i,j,0); grid.add(loc); } } }

Making ArrayList for grid

void draw(){

strokeWeight(2); stroke(0,255,255); drawSkeleton(userList[i]); drawHead(userList[i]);

background(0); context.update(); int [] userList = context.getUsers();

for(int i =0; i<userList.length; i++){ if(context.isTrackingSkeleton(userList[i])){ PVector myRHand = new PVector(); context.getJointPositionSkeleton(userList[i], SimpleOpenNI.SKEL_RIGHT_HAND, myRHand); PVector myRHand_Proj = new PVector(); context.convertRealWorldToProjective(myRHand, myRHand_Proj); PVector myLHand = new PVector(); context.getJointPositionSkeleton(userList[i], SimpleOpenNI.SKEL_LEFT_HAND, myLHand); PVector myLHand_Proj = new PVector(); context.convertRealWorldToProjective(myLHand, myLHand_Proj);

for(int j=0; j<grid.size();j++){ PVector nLoc = (PVector)grid.get(j); strokeWeight(1); stroke(0,255,0); noFill();

Get the grid point

}

All the skeleton events.

noFill(); ellipse(nLoc.x,nLoc.y,48,48);

Draw them

} }

void draw(){

strokeWeight(2); stroke(0,255,255); drawSkeleton(userList[i]); drawHead(userList[i]);

background(0); context.update(); int [] userList = context.getUsers();

for(int i =0; i<userList.length; i++){ if(context.isTrackingSkeleton(userList[i])){ PVector myRHand = new PVector(); context.getJointPositionSkeleton(userList[i], SimpleOpenNI.SKEL_RIGHT_HAND, myRHand); PVector myRHand_Proj = new PVector(); context.convertRealWorldToProjective(myRHand, myRHand_Proj); PVector myLHand = new PVector(); context.getJointPositionSkeleton(userList[i], SimpleOpenNI.SKEL_LEFT_HAND, myLHand); PVector myLHand_Proj = new PVector(); context.convertRealWorldToProjective(myLHand, myLHand_Proj);

for(int j=0; j<grid.size();j++){ PVector nLoc = (PVector)grid.get(j); strokeWeight(1); stroke(0,255,0); noFill();

} void drawSkeleton(int userId) { // to get the 3d joint data /* PVector jointPos = new PVector(); context.getJointPositionSkeleton(userId,SimpleOpenNI.SKEL_NECK,jointPos); println(jointPos); */ context.drawLimb(userId, SimpleOpenNI.SKEL_HEAD, SimpleOpenNI.SKEL_NECK); context.drawLimb(userId, SimpleOpenNI.SKEL_NECK, SimpleOpenNI.SKEL_LEFT_SHOULDER); context.drawLimb(userId, SimpleOpenNI.SKEL_LEFT_SHOULDER, SimpleOpenNI.SKEL_LEFT_ELBOW ); context.drawLimb(userId, SimpleOpenNI.SKEL_LEFT_ELBOW, SimpleOpenNI.SKEL_LEFT_HAND); context.drawLimb(userId, SimpleOpenNI.SKEL_NECK, SimpleOpenNI.SKEL_RIGHT_SHOULDER); context.drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_SHOULDER, SimpleOpenNI.SKEL_RIGHT_ELBOW ); context.drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_ELBOW, SimpleOpenNI.SKEL_RIGHT_HAND);

Make Right hand as a attraction point

context.drawLimb(userId, SimpleOpenNI.SKEL_LEFT_SHOULDER, SimpleOpenNI.SKEL_TORSO); context.drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_SHOULDER, SimpleOpenNI.SKEL_TORSO);

PVector att = new PVector(myRHand_Proj.x,myRHand_Proj.y,0); float distance = PVector.dist(nLoc,att);

Calculate distances float d = (map(distance,100,600,2,48)); Mapping radius float c = (map(distance,100,600,0,255)); Mapping colors fill(0,c,0); ellipse(nLoc.x,nLoc.y,d,d); fill(myRHand_Proj.x,myRHand_Proj.y,c); ellipse(nLoc.x,nLoc.y,d-d/5,d-d/5); }

Draw the patterns

PVector jointPosH = new PVector(); //Create a Vector (to capture the Head joint). context.getJointPositionSkeleton(userId, SimpleOpenNI.SKEL_HEAD, jointPosH); //Get the Head Joint in 3D! PVector jointPosH_Proj = new PVector(); //Create a Vector (for projecting on 2D). context.convertRealWorldToProjective(jointPosH, jointPosH_Proj); //Convert 3D Head Joint Vector to 2D screen (for projecting on 2D). fill(0,255,255); stroke(0,255,255); ellipse(jointPosH_Proj.x, jointPosH_Proj.y, 80, 80); // Extract the x and y position from the Projection Head Joint to draw a circle. }

Make Your Own Patterns