remote control gestures

Gesture technology turns everyday household objects into remote controls

New gesture-control technology is capable of turning everyday objects into remote controls and could revolutionise how we interact with televisions and other screens.

Researchers from Lancaster University have developed a technique that allows body movement, or movement of objects, to be used to interact with screens.

The technology could allow users to change the TV channel simply by moving a cup of tea, adjusting the volume on a music player by rolling a toy car or rotating a spatula to pause a cookery video on your tablet.

The ‘Matchpoint’ technology uses a webcam and works by displaying moving targets that orbit a small circular widget in the corner of the screen (pictured).

These targets correspond to different functions, such as volume, changing channel or viewing a menu. The user synchronises the direction of movement of the target with their hand, head or an object to achieve what researchers call ‘spontaneous spatial coupling’, which activates the desired function.

Unlike existing gesture-control technology, the software does not look for a specific body part it has been trained to identify, such as a hand. This new technology instead looks for rotating movement, so it doesn’t require calibration or for the software to have prior knowledge of objects.

The developers believe this system gives users more flexibility and means they do not need to learn specific commands to activate different functions, as is the case with some gesture-controlled televisions on the market, and the user is able to decouple at will.

When selecting volume adjustment or channel selection, sliders appear. The user moves their hand, head, or object in the required direction indicated by the slider to change the volume or to find the desired channel.

As well as televisions, the technology can also be used with other screens. For example, YouTube tutorials, such as mending bikes or baking cakes, could be easily paused and rewound on tablet computers without users having to put down tools or mixing bowls.

Multiple pointers can be created to allow more than one user to point at drawings or pictures on interactive whiteboards simultaneously. Matchpoint also allows users to manipulate images on whiteboards by using two hands to zoom in and out, and rotate images.

In addition to short-term couplings, users can also link stationary objects to controls, which even when left for prolonged periods will retain their control function. For example, a mug sat on a table could change a track on a music player when moved left or right or a rolling toy car could be used to adjust volume. Objects can lose their coupling with controls simply by removing them from the camera’s field of view.

Christopher Clarke, developer of the technology, said: “Spontaneous spatial coupling is a new approach to gesture control that works by matching movement instead of asking the computer to recognise a specific object.

“Our method allows for a much more user-friendly experience where you can change channels without having to put down your drink or change your position, whether that is relaxing on the sofa or standing in the kitchen following a recipe.

“Everyday objects in the house can now easily become remote controls so there are no more frantic searches for remote controls when your favourite programme is about to start on another channel and now everyone in the room has the ‘remote’. You could even change the channel with your pet cat.”

Researchers believe Matchpoint is also suitable to be used as an accessibility tool for people who are unable to use traditional pointers, such as remote controls and a mouse and keyboard.

Sign up to the E&T News e-mail to get great stories like this delivered to your inbox every day.

Recent articles