RGBD is a decent demonstration of Kinect's powers, but it's time to put the sensor to work. To do this we'll use FAAST, the Flexible Action and Articulated Skeleton Toolkit.
Developed by a team at the University of Southern California Institute for Creative Technologies, FAAST came to public attention earlier this year when it was used to turn Google's 'gesture controlled Gmail' April Fool's prank into a reality; by using OpenNI and Nite to read your skeleton, FAAST can make a decent stab of translating your movements into keyboard inputs.
Start, predictably, by downloading the latest version of the FAAST toolkit, unzipping it and running the single executable inside. If you've already installed the latest versions of OpenNI and NITE, it should run without problems. Make sure you have your Kinect sensor plugged in, select 'Upper body' under 'Skeleton mode' (we assume you're sitting down; if not, select 'Full body'), set 'Smoothing' to 0.4, then click 'Connect' to fire it up.
FAAST will start up a network server, then attempt to pick up a body. Wave at the sensor and you should be able to see your outline in the window. Now it's time for a calibration gesture to give FAAST an idea of your skeletal proportions.
In this case you'll need to perform a classic body-building pose - arms bent at the elbows, out to your sides, hands in fists pointing skyward. After a couple of seconds FAAST should work out your frame and display a simple wireframe skeleton over the top.
Movements translated
By default, FAAST is set to translate the leaning of your body into presses of the [W], [A], [S] and [D] keys - click 'Start emulator' to enable this mode and feel free to test it out in your favourite first-person game. You'll probably find it doesn't translate particularly well, but it's a proof of concept.
For better results, try closing and re-opening the application, then adjusting the smoothness value on the first tab - it ranges between 0 and 1, so use 0.x - any positive value greater than 1 will cause FAAST to stall. You'll want to experiment with this to find the perfect value for your distance from the camera and for the light level in the room.
This is impressive, but FAAST can do a lot more if you set it up to do so. You can wave your left hand around and use it as a mouse, for instance. Stop the emulator, go to the 'Mouse' tab and click 'Enable' to set mouse control as active. Your personal settings will probably differ from ours, but we found it most comfortable to use Absolute control centred on the shoulder joint, set each of the bounds to a distance of 8-inches, and the movement threshold to 5.
Start the emulator again and try it. As you move your left hand around, your mouse pointer should follow as if you're using the Force to control it.
You'll notice you're a bit trapped, though - FAAST has effectively taken over your mouse, and since you haven't defined any mouse buttons, you can't click to stop the emulator. Hit [Space], and hopefully the start/stop toggle button will trip.
Now go to the right-hand tab and enter a new gesture to bind to the left mouse button. We found something like 'left_arm_up 20 mouse_click left_button' worked a treat, although as with many things Kinect, it will increase the flailing of your arms and make you look even more like you're having a fit in front of your PC.
Kinect is rather impractical, it's true, but with the FAAST toolkit you should be able to put it to work in any number of ways.
Maximise Kinect
Do things you never thought possible
1. Learn to juggle
Download
This neat program tracks the movement of your hands and attaches glowing orbs to them whenever they're hidden behind your back. Flick your hands and you'll lob the balls in that direction. if you're skilled enough you might manage a three-ball cascade, but don't count on it.
2. Shoot a fireball
Download
Ever seen Dragon Ball Z, the slightly insane Japanese comic book and cartoon series? Run Kamehameha and you'll recognise its influence straight away: it gives you big, wobbly hair that stands on end, and lets you power up and release a world-destroying fireball if you get your poses right.
3. Render 3D objects
Download
You'll need a Mac with Processing installed to use this hack (though if you're savvy enough, you might be able to fire it up in Windows). It uses your hand-waving to sculpt objects from a kind of 3D putty, Kinect input is unlikely to be as accurate as that of a mouse, but that's not the point, is it?
4. Play with physics
Download
This demo uses the openFrameworks system instead of OpenNI, and lets you interact with gravity-centric boxes on screen. It's notable not because of the output, but because of the fact that the Kinect depth sensor removes the need for any kind of blank background.
5. Read your email
Download
The clever minds behind FAAST developed SLOOW - Software Library Optimizing Obligatory Waving - as a response to Google's 'Gesture Gmail' April Fool's prank. No actual code has been released, but it's something you could develop yourself within the toolkit. It certainly looks like fun.
6. Kung Fu Tetris
Download
Practical it isn't, but if you've cleared a big enough space in your front room you can use FAAST to have a go at a particularly physical form of Tetris. Front kicks rotate, side kicks move the pieces, and jumping straight up drops pieces quickly. Lanny Lin has kindly supplied his config file.
