Parameter name: request
Translate Request has too much data
Parameter name: request
A collection of Kinect hacks, including air guitar, animated puppets and a virtual piano
Christopher Baker spent Boxing Day as a VJ – video jockey – for a warehouse party in Liverpool. Among the items being used was a Microsoft Kinect controller, normally used to play Xbox 360 games such as Kinectimals or Kinect Sports. But Baker wasn't playing games: the system was rigged up to a computer running software to interpret the movement data being gathered by the system. The software Baker had written monitored how the clubbers were moving and used that to affect the sound, creating a feedback loop between dancers and sound. "It was popular with the clubbers because they could interact directly with the sound and were immersed in the experience," he says.
Baker, who works at Apposing, a mobile app development company based in Liverpool, is one among hundreds of people who have embraced the Kinect not for its potential in gaming (though that is substantial), but because, unusually for an Xbox accessory, its outputs can be put to different – and in some cases revolutionary – uses. In fact, the Kinect may turn out to be the device that ushers in an age of smart machines that can be rigged together and react to humans in useful ways.
Even on its own, the Kinect is an impressive piece of technology. As Microsoft's tagline goes: "You are the controller." It uses an infrared sensor with cameras to sense the movements of players in front of it. Stand in front of a Kinect linked to an Xbox 360, wave your hand and a hand icon on the screen moves too. Start playing a game and it will know if you've jumped for a spike shot in beach volleyball, hit a topspin or slice shot in table tennis or thrown a jab in boxing. No wires, nothing – the machine knows. It's an experience that is at first strange and then exhilarating.
This has made the £130 Kinect an incredible success story. It has sold 8 million units worldwide since its release in November, making it the fastest-selling consumer electronics item ever, faster than the iPad (the previous record-holder) and its predecessor, the DVD player. There's also plenty of potential for the number to grow: 50 million Xbox 360s have been sold worldwide and the Kinect will work on every one of them. Microsoft's ambitions go beyond even that: it seems that some future PC games will be controllable by it. And there are around a billion PCs on the planet.
For Microsoft, best known for its Windows and Office products, which generate billions of dollars of profit every year, the success of Kinect will be a welcome relief. Its entertainment & devices division (E&D), which makes the Xbox, had to take a $1bn write-off to manufacturing failures of the Xbox 360 and has barely bumped along in profit while sales of the console trailed Nintendo's cheaper, simpler Wii for years. But Kinect's appearance has come as E&D's profitability has solidified, even if, at $382m in the most recent quarter, it is only a 10th of that from Windows or Office. It could, in fact, turn it into a new leg of profits for Microsoft.
The Kinect solves a problem that has troubled computing experts for years: how to give machines useful vision. Ron Forbes of Microsoft's interactive entertainment business says it needed a different approach: "We knew early on [in the Kinect's development] that we had to invent a new way of approaching this problem, one that works like the human brain does." The infrared projector and sensor incorporated into the front of the device turn the world into shades of black (far away) and white (close), which it scans at 30 frames per second and analyses to find the moving parts of the image, by comparing differences between frames. These data are fed into a system tuned to recognise parts of the human body, guessing what moving part is a hand or foot or leg and estimating the movements of each.
The data are then fitted around a schema of a 20-joint skeleton (hands, wrists, elbows, shoulders, feet, ankles, knees, hips, plus two points in the lower back, and the neck and head), allowing you and a partner to be tracked in 3D, across a range of 1 to 3.5 metres, and an angle of 57 degrees horizontally and 43 degrees vertically. The Kinect also has a microphone array so it can react to voice commands.
Games designers have built sports, fitness and dance games where gestures decide what happens, but it's the "hacks", not the games, that have people like Baker so excited. The open-source software developed to interpret Kinect output – to a standard USB plug – has already been used for dozens of projects: a team at a US university has created a miniature helicopter that flies itself and avoids obstacles; another has made a "virtual piano" on the floor (you play it with your feet); the multiplayer role-playing game World of Warcraft can now be played simply by using gestures, thanks to the University of Southern California's Institute for Creative Technologies, which has written a special software "toolkit". The institute has bigger plans for the future: medical games to help people regain the use of their limbs after a stroke, indoor exercise games and so on.
Even more impressive is the work done by a team at the Massachusetts Institute of Technology (MIT), which amplified the hand detection to recognise hands and fingers and then linked it to a program for scrolling pictures, giving something of the feeling of Minority Report, the film in which Tom Cruise waved computer files and videos around in his search for crime suspects. Watching the MIT video is eerie if you have seen the film – it's as though the future is unfolding before your eyes.
Yet at first Microsoft seemed unwilling to let people dig under the surface of its software. At the launch on 4 November, it told the news site CNet: "Microsoft does not condone the modification of its products… with Kinect, Microsoft built in numerous hardware and software safeguards designed to reduce the chances of product tampering. Microsoft will continue to make advances in these types of safeguards and work closely with law enforcement and product safety groups to keep Kinect tamper-resistant."
That didn't prevent two developers and a Google engineer putting up a $4,000 bounty for anyone who could write software that would work independently of the Xbox and so let it be used for more than just games. As they pointed out, people had done much the same with Nintendo's Wii, which uses the widely available Bluetooth communications standard, to create some interesting hacks.
Within a fortnight, Alex Kipman, the Xbox's "director of incubation", was insisting the Kinect was open by design and that Microsoft was excited about the idea of people designing new applications for it; the company seems to have initially misunderstood what sort of hacking was going on and thought that people were opening up the innards of the machine to get at its proprietary (and highly valuable) chips and software, rather than simply playing around with its output.
At that, the floodgates opened. By 14 December, there were "official" open-source drivers – the essential software that lets a computer understand what the stream of incoming data refers to – and the list of hacks is growing almost by the hour.
Baker, who has a longstanding interest in graphic design and motion graphics, is thrilled at its potential: "What would have once taken weeks of careful editing can now be generated live with software that costs the user little more than some of their own time and its patient application."
And the future? The sky's the limit, Baker suggests. "Gestural interface will change the way we use information devices for the young, old and disabled and allow interactivity to take place in locations where it would once have been impossible."
There remain plenty of new areas that the Kinect could be used in. Microsoft is still keeping its voice detection system under wraps – this seems to be done inside the Xbox 360 – which has dismayed hackers who want to get closer to it.
The most encouraging thing about the Kinect for the wider use of computing is that it is not the result of a surge in computer processing power. The Xbox 360 first appeared in late 2005 and that's "veteran" by today's computing standards. Microsoft says the processing required for Kinect takes less than 10% of the Xbox's computing power. This means that running the Kinect takes comparatively little computing power by modern standards, so its technology could be incorporated into machines of the future, which would be able to recognise us (and identify us by the distances between our joints), watch us moving around and react appropriately.
Imagine an old people's home where Kinects could spot someone falling over or someone who hadn't moved for a long time. Or perhaps a door-opening system that would react to your gestures to open it: as it recognises an individual's unique 20-joint shape, only the right person making the right gesture gets in.
The possibilities are endless. And for Microsoft, it's all profit.
The title of "fastest-selling gadget" is not easily won. Before the Kinect deservedly took the crown at the end of 2010, with 8 million sold in around 50 days, the previous holder was Apple's iPad. It went on sale last April and sold 3m units in its first 80 days, and 7.4m by the end of September, the last announced results from Apple.
But 2010 was unusual in seeing the title change not once but twice; before that it had been a long time between crownings.
In fact the previous record-holder for the title was the DVD player, which sold 350,000 units in its first year, 1996. That means twice as many Kinects were sold each week as DVDs were in an entire year.
It is difficult, however, to be absolutely certain about these technological milestones; the Guinness Book of World Records doesn't officially track sales figures, and some companies simply don't release their numbers.
And expectations for what will be the best-sellers are often dashed. In October 2007, Comet forecast that high-definition DVDs would be best-sellers. Five months later the range was discontinued – having lost out to Blu-ray.
' Not what It says on the tin – ie, make people.» Photos: Simon Stuart-Miller
Apple iPads and iPhones will transform how we get our news in the future – but on the other hand ... Photo: Tim Ireland/PA
Colin Firth arrives with his wife, Livia Giuggioli to Golden Globe Awards last week. Photo: Matt Sayles/AP
Stacking ... an adventure game like no other
Public data corporation announced by Francis Maude. Photo: Martin Argles/guardian