Nyanekt and other Kinect Experiments

I had been playing around with the Kinect for a little while, but had never really made it past tweaking the basic examples. My primary achievement had been a greyscale depth-as-value rendering of a point cloud. Not too exciting.

This weekend, I sat down and dug a bit farther into OpenNI, which is a great library for skeleton tracking. I had the OpenFrameworks ofxOpenNI examples up and running, but learning oF drawing libraries seemed like a bit of a daunting task. Instead, I made use of the ofxOSC library to port over skeleton data to Processing. My first success was with the sketch below, which is basically a hand-tracking sketch connecting the endpoints of lines to the center of the sketch. The colors are based on the depth values of both hands, with one colored as (leftHand.Z, rightHand.Z, 255) and the other as (rightHand.Z, leftHand.Z, 255). It’s simple, but I think it turned out nicely.

The real success of the evening, however, was producing the sketch at the top of this post, which emulates Nyan Cat with Kinect data. The cat is positioned according to the user’s hips, which is basically an attempt at making people dance out the Nyan Cat. Once I had an OSC stream of hip coordinates, it was relatively simple to stitch those coordinates together in to the ribbon (ribbon displays the last 800ms of activity). As a final touch, I added both a left-facing and a right-facing cat, to go along with which direction the user was moving. All that’s left to do is add some NYAN NYAN NYAN audio output!