Interactive with Kinect

submission1  Dora SHEN

 

The project is called Environments in (e)motion, “environment” certainly refers to the specific site (Sculpture Court in ECA main building), and “emotion”, from my point of view, is to arouse the deep feelings of the visitors. However, when it is written in a special way with a pair of parentheses, it is more like creatively encouraging the visitors’ physical motions.

In this light, the ideal device to produce physical interaction for this project is a Kinect. As a motion sensor, Kinect can be used to detect gestures and to make interactive effects if working with other equipment and software (i.e. Processing, projectors, laptops).

devices

So we decided to make an interactive projection on the wall in the Sculpture Court. The content will be based on our concept and immerse into the atmosphere we are going to produce. In the first submission, what we have done is basically to borrow the sensor and set the environment on the laptop for it to run, and to do some tests by running sample sketches.

So far, by using Kinect to capture motions, we can project a human skeleton onto the wall, fill in colors, pixelise the human body, or generate floating geometric figures around the outline of the human body. The immersive effect would be much better after we create a sketch that is more related to the core concept and environment settings. However, these interesting motion interactions could already encourage participations and impress the visitors. In fact, when we were testing the devices in the Sculpture Court, people who were accidentally passing by showed their interests and took out their phones to take pictures. We felt encouraged after seeing them interested in our project, even we were only doing tests in a corner upstairs!

 

IMG_6232

testing Kinect

IMG_6233

Kincet Pixelization

Reference

Alex Bradbury-Wallad. 17/02/2015. OpenProcessing Accessed from <www.openprocessing.org/sketch/186336&gt;

Amnon Owed. 15/09/2012. Accessed from <www.creativeapplications.net/processing/kinect-physics-tutorial-for-processing/&gt;

 


Here’s some technical issues concerning to getting Kinect and Processing to work together. It is already half way through the DMSP project, and I have encountered lots of problems during the whole procedure, here I list all the problems as well as corresponding solutions. The idea undoubtedly becomes clearer during the problem-solving process.

 

Setup Kinect

  • PC –  Only Kinect 360 (version one) can be borrowed from ECA music store. For a PC laptop, the drivers that are required to get it to work are Kinect for Windows SDK v1.8 and Kinect for Windows Developer Toolkit, which can both be download on the Microsoft official web site: www.microsoft.com/en-us/download/details.aspx?id=40278. System requirements and install instructions are well elaborated here.
  • Mac –  Since Kinect was developed by Microsoft, there are less support on Mac OS X, lots of additions and dependences are required, such as Xcode (as well as Command Line Tools), CQuartz, CMake, MacPorts, OpenNi SDK, SensorKinect and NiTE. Also several commands should be typed into terminal to finish activations.

 

Processing libraries

  • SimpleOpenNI (recommended) –  Among all the libraries designed to work with Kinect, I think SimpleOpenNI is the most powerful. It is widely used to play with Kinect, so many helpful tutorials and resources can be found on the internet. Also there are solutions to various problems while designing interactive sketches with Kinect in the Processing forum, which cover most of the questions a beginner might ask.
  • SimpleOpenNI can only be applied on Processing version 2 and before. Notice that there are two main versions: 0.27 (2012) and 1.96 (2013), most examples on the internet is written based on version 0.27. So we need to do some “translation” work for it to run on a version 1.96 library, the keyword update such as enableScene to enableUser, sceneImage to userImage, enableHands to enableHand, addGesture to startGesture, removeGesture to endGesture.
  • OpenKinect –  a library that always reports errors. Use Zadig to install the libusbk driver for each device people wish to use. This includes installing the libusbk for all the Kinect 360 drivers listed on Zadig GUI.
  • PKinect –  not so widely used.

 

Leave a Reply