Build Projects Using the Kinect

The Kinect Sensor::

Detailed Description of the Kinect Sensor
Wikipedia Definition

The Kinect contains three main components that operate together to detect motion and to represent physical images (i.e bodies) on the screen: an RGB color VGA video camera, a depth sensor, and a multi-array microphone.

  • The camera detects the red, green, and blue color components as well as body-type and facial features. It has a pixel resolution of 640×480 and a frame rate of 30 fps. This helps in facial recognition and body recognition.
  • The depth sensor contains a monochrome CMOS sensor and infrared projector that help create the 3D imagery throughout the room. It also measures the distance of each point of a person’s body by transmitting invisible near-infrared light and measuring its “time of flight” after it reflects off the objects.
  • The microphone is actually an array of four microphones that can isolate the voices of a person from other background noises allowing people to use their voices as an added control feature.

These components come together to detect and track up to 48 different points on each player’s body and repeats 30 times every second.

Would you like to use the Kinect in projects that use MaxMsp, Processing, OpenFrameworks etc..?
Then read on for detailed instructions on how to set up your environment::

Please Note: The instructions that follow have been implemented and tested on Mac OS X: 8 (Mountain Lion) and 9 (Mavericks) ONLY:

  1. Download and install the latest version of Xcode from the app store. Once it is installed, open up Xcode and select:
    Xcode > Open Developer Tool > More Developer Tools..
    Sign in using your AppleID (if you don’t have one, register, it’s free).
    Download the Command Line Tools for Xcode and install it.
  2. Download the CMake utility (last tested with version 3.05 on Yosemite).
    Once downloaded – install the utility by following the on-screen instructions —
    You may get an “unidentified developer” prompt from OSX when you try to install CMake:
    SO::
    Select OK. Navigate to the pkg and right click and select OPEN:
    You shall see another prompt, ignore and select Open:
    Now install. NB:: when installing please remember select the option to install command line tools as well!
  3. Download the macports utility application. Again follow the on-screen instructions to install the application on your computer.
  4. Once macports is installed we will launch this application using the terminal.
    On your machine open the terminal by navigating to Applications/Utilities/Terminal. Double-click, and a window should open…
  5. We will now install libtool through macports:
    In the terminal window, please type in the following:
    “sudo port install libtool” & press ENTER.
    Please type exactly that command – the terminal will ask you for your computer password (if you have one) type it in (cursor will not move) and press ENTER.
    If successful, the terminal will download and install the libtool utility onto your machine. It will take a few seconds – and the terminal will let you know when its done (do not close the terminal before it has completed …
  6. Next:: we will install libusb through macports – again in the TERMINAL type:
    “sudo port install libusb +universal” & press ENTER.
    Please type exactly that command – the terminal will ask you for your computer password (if you have one) type it in (cursor will not move) and press ENTER.
    If successful, the terminal will download and install the libusb library onto your machine. It will take a few seconds – and the terminal will let you know when its done (do not close the terminal before it has completed …
  7. Next:: Install git through macports – again in the TERMINAL type:
    “sudo port install git-core” & press ENTER.
    Please type exactly that command – the terminal will ask you for your computer password (if you have one) type it in (cursor will not move) and press ENTER.
    If successful, the terminal will download and install git library onto your machine. It will take a few seconds – and the terminal will let you know when its done (do not close the terminal before it has completed …
  8. We are now ready to download, build and install libfreenect . This is the library that will allow for an application (MaxMsp, OpenFrameworks …) to communicate with the Kinect Sensor. We will be once again be using the Terminal application (Applications/Utilities/Terminal) to perform the necessary steps. If you need any information or further help/instructions regarding libfreenect: please follow this link:
    https://github.com/OpenKinect/libfreenect
    We will now download libfreenect using git in the TERMINAL – Applications/Utilities/Terminal:
    In the terminal window first navigate to your home folder by typing:
    cd /Users/’Your username’
    Next: we want to make a new directory called “libfreenect” so type the following:
    mkdir libfreenect
    Go into this new dir by typing:
    cd libfreenect
    Now we want to download the libfreenect files using git. Thus, in your Terminal type in the following command:
    git clone https://github.com/OpenKinect/libfreenect
    This command will place the libfreenect files in /Users/’your_user_name’/libfreenect.
  9. We need to now BUILD libfreenect (so far we just downloaded the files):
    Once again, in the Terminal window type in the following command:
    ccmake /Users/’your_user_name’/libfreenect
    You may encounter and EMPTY CACHE warning, simply ignore and select the ‘c’ key to move onto the configure screen:
    This will result in opening the configuration screen. You may edit configuration variables, but here we are just doing a simple installation. The most important thing about this screen is to check the two vars LIBUSB_1_INCLUDE_DIR and LIBUSB_1_LIBRARY. Make sure they pointing to the include folder and libusb-1.0.dylib file, like so:
    LIBUSB_1_INCLUDE_DIR: /usr/local/include
    LIBUSB_1_LIBRARY: /usr/local/lib/libusb-1.0.dylib

    If all checks out, press the ‘c’ key twice or until you get Press ‘g’ to generate and exit command.
    Select the ‘g’ key. This will close the configuration window and finally proceed by typing in the terminal:
    make
    This will output a bunch of text which mainly scans for dependencies, links executables and builds:
  10. Finally: In the Terminal Window type the following command:
    sudo make install
    You will be asked for admin password – type it in and press ENTER – then:
    This command SHOULD successfully install libfreenect on the system path…

And now you should be all set to use the Kinect Sensor!
The easiest way to test your Kinect sensor is to use the examples that come with libfreenect. To use these examples:

  1. Plug in the Kinect hardware to your Mac usb port and connect it to power.
  2. Navigate to the libfreenect examples by typing the following command in your TERMINAL app:
    cd /Users/’your username/libfreenect/bin’
  3. Next we want to view the contents of this dir, so type:
    ls
  4. You should now see a listing of different examples – so if we want to run i.e. “freenect-glview” then type:
    ./freenect-glview
    Your Kinect sensor should now respond – according to the instructions output in the terminal….

To setup a Kinect project in OpenFrameworks on a Mac OS machine:

  1. First download OpenFrameworks.
    Once installed – please refer to the Setup Guide for how to develop your projects within the Xcode environment.
  2. Within OpenFrameworks – there is already an existing Kinect Project that will allow you to first test the Kinect. You can then extend/ change the example project and make it your own …
    To find the example on your machine – go to your openframeworks folder and navigate to addons/kinectExample/kinectExample.xcodeproj. Follow the setup guide (see link above) for running the example in XCode.

To setup a Kinect project in Processing:
The following online article by Daniel Shiffman: Kinect & Processing Link gives detailed instructions for developing Processing projects with the Kinect Sensor.

To setup a Kinect project in MaxMsp:
Finally: if you like MaxMsp & Jitter: there is a super patch developed by Jean-Marc Pelletier freely available on his webpage. Just download the patch + help patch (tested in MaxMSP 6) and run the help patch – again you can extend/change the existing patches to meet the needs of your project…