Kinect+VVVV Interaction Design

Kinect + VVVV Interaction Design

Hongik Univ. IDAS 2016 Media Environment Design Studio II Professor David Hall

Digital Media Design Jong-won Lee

appsdesigncom@gmail.com

Step #01

Idea Developments

I got interests in communication between the physical and digital. So I started making many ideas as below;

idea 1 : Project a big game character with a big and powerful projector on the street. Then the street becomes a game stage. But how can the game player hit by a car or a bus?

Idea 2 : Making a maze with Lego, then pr Then this becomes a physical maz

Idea 4 : If I use Vuforia VuMarker, I don’t need to make mazes physically.

Then I just need to put registered sticker on the wall and desk, it will be presented as mazes of its kind.

Idea 5 : If I use projector to project the Vuf then it can be controlled as I want

roject the game on the maze. ze + a digital game.

foria VuMark Stickers on the wall, ted to make.

Idea 3 : Make more Lego mazes : one Lego on the desk, one drawing on the wall, one Lego on the wall

Step #02

Project a 2D game on the LEGO building

I made one simple Unity 2D game and project it on the LEGO building. This is made for prototyping purpose.

I just use some walking sprites from Super Mario game by Nintendo.

Step #03

Project a Vuforia VuMark-type Image-Target on the wall

I also made and try the Augumented Reality way of communicating between the real world and digital device.

By using Vuforia, it was very easy to test it.

Step #03

Evaluate the Prototypes

It is easy to project on the fixed object. But it would be whole alot different when it starts to move.

By using the Arduino and stepper motor, it starts rotating.

Step #04

Generative Arts & Openframeworks

I also tried new digital design tools or area : the Generative Arts. It can be said that ‘designing the logic or algorythim to let the machine do the design that I want. But the result is somewhat unpredictable, but automated way.’ ‘

This is the random shapes by coding on the Openframeworks.

I checked the Processing, P5.js, Openframeworks, MAX/MSP, VVVV. The Processing and P5.js is mostly code based one. And Openframeworks consists lots of Addons and librarys, so it is alot easier and has more capabilities and possibilities. But I also have to study C++.

Random box shapes generation on the screen (Openframeworks)

Step #05

Openframeworks : Kaleidoscope-like shape shifter

By following the example on the book ‘ddd’, I could achieve to make this Kaleidoscope-like shape shifter. This is programmed on macOS by XCode.

Step #05

Openframeworks : Opportunity & Possibility

It can handle graphics much better than the Processing or just code base programs. Here are some examples on its examples library. You can see its fast and accurate rendering of the graphics.

Step #06

Openframeworks : Visual camera & Depth Camera test on the Kinect

I also tried the Kinect by Microsoft on the Openframeworks. There were examples for the Kinect, so it was easy to test the Kinect connection and its working.

As you see above, you can see the Visual Camera, the Depth Camera mode is ON.

Step #06

VVVV + Kinect : Connecting the body movement and shape shifting control.

But it was hard to code on the Openframeworks for beginners. Then I decided to use the VVVV to make the project.

It is [PATCH] - [NODE] - [PATCH] connect method. I can not say it is easy, but a little bit easier then coding only type programs like the Processing.

I could find the Kinect Patch and Skeleton information on it. I also could find the shape shifting patch. I studied its structures and made it by connecting its patches with cables (nodes).

Step #07

VVVV + Kinect : Test (1) Myself

I setup the stage with the Projection, the Kinect, Windows Notebook to test.

It was much more interesting than I expected. I also could see that my shadow on the projection screen have some meaning. I felt that it is another kind of my reflected-self image, but not 100% myself. If I can control my shadow and if it communicate with the program, it would be great.

Step #08

VVVV + Kinect : Test (2) Nico

Classmate Nico tried to test it. He also liked the movements of the shapes.

Reference

1. Openframeworks (http://openframeworks.cc) :

1) OpenFrameworks Essentials [Denis Perevalov, Igor Tatarnikov / Packt Publishing (2015)]

2. VVVV (http://vvvv.org) :

1) Prototyping Interfaces [Von Jan Barth, Roman Stefan Gracy, Mark Lukas / Schmidt (Hermann), Mainz (2013)] 2) Generative Design [ Hartmut Bohnacker, Benedikt Gross, Julia Laub / Princeton Architectural Press (2012)]

3. Vuforia (http://vuforia.com)