Motion control has come on in leaps and bounds (and swipes and punches) in recent years, and hardware companies are getting all e-motional in a bid to engage with users. But do we really know what to use it for yet?
Smartphones and tablets have led us into a touchable digital world. Gone are the days of pressing keys and moving a scroll wheel. Now we expect instant feedback via touch controls and air-based gestures are the next natural step for this type of interaction.
Companies like Leap Motion are looking to make waves in the human-computer interface market with a product that enables users to manipulate on-screen items not with a joystick or a mouse or a gadget of any kind, but directly with hand gestures.
Last year, Hewlett Packard announced an expanded partnership with Leap Motion that saw gesture-based controllers embedded in a range of HP laptops. And with Apple patenting a touchless control interface, it's time for manufacturers to start thinking about the next steps in user interaction.
Norwegian startup Elliptic Labs has made an ultrasound software development kit (SDK) that uses sound waves to interpret hand movements. Then there's PrimeSense, the company behind gesture control for Microsoft's Xbox Kinect, which recently demonstrated a smaller version of the Kinect sensor working with a Nexus 10 tablet at a Google developers' conference in May.
However, it's Leap Motion that has arguably the biggest potential to reach users.
Founded as OcuSpec in 2010, Leap Motion has amassed a global developer base tasked with creating apps for its gesture-based controller, which can be bought as a plug-and-play USB device or picked up as an embedded peripheral in laptops and keyboards.
CEO and co-founder Michael Buckwald has been busy promoting the application store where several developers have created applications for the device.
"One of the things we've done over the past few months is create a new module reference design that takes the module at the top of the peripheral, which is about 10mm, and reduce it to 3.5mm so that it's easy to be embedded in things like laptops and ultra-thin keyboards," he says. The company expects to launch a version for tablets and smartphones later this year.
"We've been saying for quite a while that our goal is to have Leap Motion in everything that's a computer or has a computer, so I think that tablets and phones are a natural next step. Our conversations are ongoing, but that could happen as early as this year – not only integration into tablets and phones but we'd also like to see additional orignal-equipment manucaturing (OEM) partners on the PC side," adds Buckwald.
In fact Buckwald sees professional applications as an important growth area on the app store. He explains how he was frustrated with the fact that a toddler can make something with modelling clay, but it takes a professional engineer hours to make a model using professional tools. "There are lots of examples of that, whether it's things in 3D or creation sculpting, education or exploration," he says. "Those experiences just don't happen on computers today, and our mission is to bring them to PC and to bring them to tablets."
Leap Motion's flagship product Airspace comes bundled with units such as the HP Envy 17 Leap Motion Edition laptop which is likely to encourage developer growth, especially as HP is implementing Leap in 11 stock-keeping units (SKUs) or different versions of the products – those being desktop and all-in-one computers – bringing the total number of SKUs to 12. HP has embedded Leap Motion technology into a keyboard that's included with 11 new SKUs,
One of the company's developers is Autodesk. It has a plugin for Maya,which is a $5,000 to $10,000 suite used by software engineers. Another is the Cubase iC pro music app, and Leap Motion is even being implemented in surgeries and other environments to attempt to reduce the possibility of infection.
Wand TV control
At the annual Consumer Electronics Show Las Vegas in January, Samsung took the wraps off its brand-new TV remote control. It has every control method imaginable baked in, from buttons to a touch pad to motion control and even voice recognition.
"As more and more functions are added to TVs, the remote control is also evolving to accommodate the added functions," says KwangKi Park, Samsung's executive vice president of visual display sales and marketing. "We will continue to work so that our customers can use remote control more intuitively and easily."
Other features on the new remote include'a''Soccer Mode' button, which apparently flicks your TV over to a set of interface and screen modes custom-built to best show off the football, and 'Multi-Link Screen' which is a fancy picture-in-picture mode. The new controller was on display with Samsung's new 110in Ultra HD TV at the show.
As ever, Samsung will want to steal a march on any motion-control interface that rival Apple may introduce in the near future. But motion control has been touted for TV for several years. In fact, Philips has long been demonstrating a gesture control device – but so far there has been little uptake by other TV manufacturers of their technology.
Television remote controls haven't changed a great deal since they were invented. The first generation looked more or less as they do now, but were often connected to your equipment with a cable. Philips's idea is a gesture-control device called uWand. It's exactly that: a wand, which you use to move through your TV's menu systems.
It is reminiscent of the Nintendo Wii remote, with the difference being that the Wii controller is designed for playing games – something that's obvious when you try to use it to press buttons in the console's menus, which is fiddly and annoying. The uWand is much smoother. The cursor doesn't jump about all over the place, and grabbing hold of things in the menus is simple.
Perhaps now is finally the right time for the uWand. Modern televisions are dominated by menu systems, especially for using the programme guides. On a standard remote, navigating through listings requires countless button presses, and is quite a pain to use. The uWand simplifies that by letting you scroll with a twist of the controller and select items to see in more detail with a large 'OK' button, one of only three physical controls on the remote.
What's more, you can record a programme by dragging the name of the show from the EPG to a drop zone. This makes recording things much faster than messing around with arrow keys and buttons.
Early versions of the controller employed a Wii-style sensor bar, which feeds back information to the remote about positioning. Philips says that when it makes its way into TVs, it will simply be integrated into the bezel. The remote communicates via radio frequency at the moment, but Bluetooth connectivity is in the pipeline.
Philips won't be keeping this technology to itself; any TV manufacturer can license it, with the hope that viewers will soon be able to interact with their new TVs in an entirely new and more logical manner. Whilst Philips has developed the hardware, user interface design will be handled by the TV manufacturers. Philips is also keen to point out that the uWand is well-suited to navigating around a 3D environment too.
All the gesture and motion control technologies currently being promoted are line-of-sight or gyroscopic based – except one. Elliptic Labs is a Norwegian based company that has developed gesture recognition technology using ultrasound for touch-less gesturing smartphones and Windows PCs.
Laila Danielsen, the company's CEO says its software "delivers touchless gesturing in a natural way all around the screen of a smart phone, tablet or laptop at 180 degrees, using the movements you use in daily life.
"It enhances the way you interact with applications, such as browsing through pictures, social media and games like Fruit Ninja or Subway Surfer; it uses little power, and works in low light or in the dark."
The software can recognise gestures within 180 degrees from the front of a display at a range of 50cm to a metre away from it. Developers or users can create their own gestures in ways that help them navigate quickly.
The Elliptic SDK is currently being licensed to mobile phone ODMs in collaboration with Wolfson Microelectronics for use with its Audio Hub chip and MEMs microphone technology operating at around 40 KHz – although it can operate at higher and lower frequencies – explains Haakon Bryhni, chief technical officer, for Elliptic Labs who is based at the company's Norwegian R&D centre.
"Along with voice control, touchless gesture control is fast becoming one of the'next-generation human/machine interfaces for mobile and wearable devices," says Andy Brennan, commercial director for Wolfson.
The main push for Wolfson and Elliptic appear to be mobile. At the upcoming Mobile World Congress, Elliptic Labs will be present and will demonstrate its technology to potential original device manufacturers (ODMs) at the mobile industry's annual tech jamboree. No mobile phone company has currently announced that it is to incorporate the Wolfson/Elliptic Lab touchless gesture platform, but Danielsen claims that we will see devices with their technology later this year.
Gesture controls certainly have the potential to improve the existing interface experience, and there is a huge opportunity for a number of tech companies to define the future of gesture controls. The marketplace is ready and waiting for that one killer application to take us to the next level of human/interface interaction.