The Kinect’s cocktail of computer creativity has proven irresistible to developers, gamers, scientists, engineers and teenage techies alike who are borrowing its technology to create new applications that encompass everything from household appliances to robotics to surgeons tools.
Within 60 days of its launch last November Microsoft’s Kinect motion controller for the Xbox 360 had shifted a massive eight million units and garnered the Guinness World Record for fastest selling consumer electronics device. Even more unprecedented however, is that the Kinect has also sparked a technological revolution that goes way beyond just playing games.
The Kinect peripheral has an RGB camera, depth sensor and multi-array microphone running proprietary software, which provide 3D motion capture, facial and voice recognition capabilities. This means the user can control and interact with the Xbox 360 using gestures and spoken commands - without the inconvenience of actually having to pick up a controller. In other words the user becomes the controller. But speedier than even the Kinect’s sales, techies around the world have seized on its attributes to develop their own applications.
Rather than encrypting raw data from the Kinect to prevent ‘hacking’, Microsoft has by design encouraged the hacker hoards, recently launching Kinect Fun Labs, an online community that allows innovations to be shared.
By the same token PrimeSense, the Israeli company that developed the Kinect’s 3D technology, has been equally as forward thinking, allowing access to its OpenNI software which reduces reams of raw data from the sensors down to “easily manipulated skeletons” of what has been recorded. And there are now several websites dedicated specifically to monitoring the latest uses for the device.
The key to this technological transformation is cost. Amazingly for such a sophisticated sensor it’s comparatively cheap – retailing at just over £100 rather than the weighty bank loan required for many previous models. Meaning that for many, concepts that previously would have remained fantasy can now easily be realised.
“We’ve seen so much innovation and creativity by people around the world in building things with Kinect,” Kudo Tsunoda, creative director for the accessory said recently. “Developers have been experimenting in ways we could never have envisaged.”
Here are the top five Kinect hacks that Microsoft never saw coming.
Viewing patient MRI and CT scans
Engineers at the University of Toronto and surgeons at the city’s Sunnybrook Health Sciences Centre have hooked the Kinect system to a computer in the operating room, which allows surgical staff to more intricately control and view MRI and CT scans by a simple wave of the hand. The system uses 3D data from the Kinect to spot the surgeon and follow his or her gestures - allowing the surgeon to control the computer’s programs without ever touching it or leaving the sterile area. “This is pure magic for the operating room,” extols Dr Calvin Law, surgical oncologist at Sunnybrook’s Cancer Centre.
Motion capture in CGI
Creating the believable blue skinned Na’vi humanoids on their lush planet Pandora in James Cameron’s Avatar required a special sensor suit for motion capture, umpteen cameras, high-end computer wizardry - and mega bucks. But now creative collective Triangle Productions are posing a serious threat to Cameron - for less than the price of his lunch. They’ve used the £120 Kinect to build a motion-capture interface for their animated CGI online cartoon series Under the HUD. Triangle uses the Kinect to control the motions of their CGI "puppets" instead of employing digital artists to painstakingly manipulate the simulated joints and limbs of the 3D creations. In turn this system can be used as a replacement for motion-sampling – the technique used in Avatar.
Force feedback during robotic surgery
Robotic tools have long been used for minimally invasive surgeries. They usually require complex joysticks to control surgical instruments and cameras inside the patient and there is no way for a surgeon to ‘feel’ where he is going. If an instrument hits something solid it will stop but the joystick will keep moving. Electrical engineering students at the University of Washington have solved this problem by writing code allowing the Kinect to map and react to environments in 3D - and send the information back to the surgeon. So if the surgeon bumps into a bone the joystick will stop moving. The system also allows for creating ‘force-fields’ around vital organs protecting them from incoming scalpels. Crucially before the advent of the Kinect a similar system would have cost around US$50,000.
Home lighting control
A US tech whizz known as Nitrogen has developed the Kinect to control the lighting system in his apartment. By placing the Kinect box in the corner of a room to give it the widest possible view and connecting it to an automation controller with software that tracks movement around the room, lights switch on and off or dim according to his whereabouts. As the Kinect detects presence and motion the lights won’t switch off when he remains still. It also works with more than one person in the room. The advantage of the Kinect over a standard occupancy sensor is that when connected to an automation system it can be controlled from anywhere in the world with an internet connection - and it saves energy.
Navigation systems for the blind
The University of Konstanz in Germany has created NAVI (Navigational Aids for the Visually Impaired) that provides blind and partially sighted people with better awareness of indoor surroundings. The Kinect sits on top of the person's head attached to helmet and will look around for a route and, depending how far away it is from a marker, give instructions like “turn right, watch out for obstacles on the left, door ahead in 3,2,1…” At present the NAVI also involves a lot of accessory instrumentation to be strapped to the person but even at this early stage it looks set to be one of the most useful systems of the future.