All in the mind

Thought-control was once the domain of big-budget military projects but that's no longer the case, explains E&T.

This is probably the last thing you want to be thinking about right now but Christmas is coming. If you've got kids they may already have informed you of the impending availability of a range of games based on 'thought control', whether it be the Star Wars Force Trainer from Uncle Milton, Mattel's MindFlex or, for video gamers, the EPOC headset from Emotiv or the impressively named Neural Impulse Actuator (NIA) from OCZ Technology.

What they all have in common - at least according to their publicity - is the ability to sense the signals generated by brainwaves, filter and process them, then use them to control a device or computer. They all feature wireless headsets, and in the case of Force Trainer the aim is to control a ball in a fan-driven tower, while in MindFlex the ball has to be manoeuvred through a fan-operated obstacle course.

Both rely on technology licensed from biosensor company NeuroSky to capture brain activity according to whether the user is concentrating or relaxing. And both are due out in the US in time for Christmas - Force Trainer for $130, MindFlex for $80 - although neither has a definite date yet for European release.

EPOC is a wireless headset that detects 30 or so different emotions and their associated muscle movements, such as anger (eyebrows furrowed). The headset is aimed initially at controlling video games on Windows PCs, but makers hope in time it will be able to control any home electronics device. Expected to cost about $300, it has yet to be released commercially, although for R&D purposes headsets with software development kits are available now.

The NIA, however, has been available for about two years and, like EPOC, it too is aimed at Windows PC games and captures a mixture of muscle, skin and nerve activity but connects via a USB 2.0 port. Current price is about $100.

All four are marketed as a brain computer interface, or BCI (although the term brain machine interface, BMI, crops up elsewhere). To a casual observer, therefore, the challenge of producing a cheap and practical BCI would appear to have been solved. Not so, however, because these are not true 'thought control' devices in the sense that BCI researchers at the serious - principally medical - end of the technology see it.

Types of brain control

"Broadly, BCIs rely on two types of responses to work - being able to measure a brain response in response to an external stimulus and/or being able to measure a response following 'self-generated' actions, as in 'imagine moving your right arm' or 'relax', for example," explains Dr Christopher James, co-leader of the University of Southampton's BCI research programme.

"These games tend to fall into the latter category of responses, which are generally harder to measure and make sense of, because if you ask 10 people to do the same thing they will all do it slightly differently, giving variations in signal frequency and amplitude. Also, because these are games, siting of the electrodes will not be perfect and contact will probably not be ideal either, so the signal-to-noise characteristics will vary between users.

"Then add to that the artefacts [spurious signals] generated by movements of muscles, electrodes and so on, and the reality is that the signal actually being measured is so contaminated that whatever is being processed is not really influenced by brain activity but rather the artefacts - which can be hundreds of times larger than the actual brain activity," he says.

"You may argue that they are still exerting control - which would be true - but that would not really be a BCI system. Some could argue the brain is controlling the muscles, hence it is BCI, but that is a weak argument from our field's point of view."

James concedes that what these games show is that it's possible to measure brain electrical activity and attribute that activity to a known action on the subject's part, that it's possible to do that using equipment that can be manufactured cheaply and mass produced, and that BCI can be as simple as asking you to "empty your mind".

"These are all good things and can only help in the 'serious' business of BCI," he says. "But the danger is that people get carried away. For example, I can ask you to empty your mind or imagine moving your right arm, as I know what sort of response to expect and look for. But I can't ask you to think of a colour and be able to guess it - I can show you a bunch of colours in random order, ask you to think of one, look at them on the screen then figure out which one you're thinking of, but I need your cooperation in doing so.

"The games are a bit of a parlour trick - but as long as they are explained for what they are then they could put BCI in a good light. But no way is BCI a done deal - a practical BCI in an application that counts still needs research."

Invasive BCI measurement

This research falls across the three principal methods of measuring brain activity - non-invasive BCI, using electrodes placed separately on the scalp; partially invasive, via an electrode grid that sits inside the skull but on the surface of the brain; and invasive, where the grid is implanted into the grey matter itself. Most current research is divided between the more established invasive and non-invasive approaches, although partially invasive techniques hold promise as an intermediary.

Invasive BCIs produce the best-quality signals, as they can be implanted in the exact area of interest and the signals are not jumbled up with artefacts. But the fact that they are invasive means surgery is needed to fit them, creating greater technological demands and risks of infection, although there are those who dispute these are major issues (see 'Implants vs EEGs, opposite).

They also measure activity in only one area of the brain, and at the moment their signals weaken with time, because of scar-tissue build-up around the implant as the body reacts to the invasion.

Nonetheless, this is an area of intense research, most of which is being conducted in North America. One implant-based system is BrainGate, developed by biotech company Cyberkinetics in conjunction with the neuroscience department of Brown University in Rhode Island. At this stage it's only an investigational device and is undergoing clinical trials.

Designed initially to help people who have lost control of their limbs but whose neurons responsible for these motor functions are still active, the system includes a silicon-array sensor a few millimetres square containing about 100 electrodes implanted in the brain's motor cortex. The electrodes are wired to a pedestal mounted through the skull, and the pedestal is connected by a cable to a computer.

First patient

The method was first tested on a human patient - a tetraplegic - in 2005 to move a computer cursor, change the volume or channel on a TV set and control a robotic arm. The BrainGate team is also looking at providing the ability to control devices that allow breathing, bladder and bowel movements, with the eventual aim of 'rewiring' the nervous system to enable patients to use their own limbs again. The team is also developing a self-contained wireless version of the implant.

Non-invasive BCIs, which are principally electroencephalographs (EEGs) - the technology used in the new games and headsets - are cheaper, easily replaceable and in multichannel arrangements can detect signals from more than one area of the brain. But the electrodes usually need to be kept wet to give a good electrical contact (although they're dry in the games versions) and even then can become detached.

Their signals are also more complex as they're mixed in with those from artefacts and distorted by the brain and skull, and even suffer contamination by the 50Hz mains. They therefore also have a poorer signal-to-noise ratio.

But EEG technology has enabled researchers to unveil 'real-world' prototypes more quickly. For example, Toyota and Milan's Polytechnical Institute have each demonstrated working versions of thought-controlled wheelchairs, while Honda has unveiled a BMI to control its Asimo robot, although this is not via EEG alone.

Toyota says its BMI works almost in real time - responding to the user's thoughts in 125ms rather than the typical time of several seconds - and that the   system can adjust itself to the characteristics of different users so that it quickly learns their directional commands, with a claimed accuracy of 95 per cent.

In the Milan version, the user sends a signal down a cable to a laptop by concentrating for a few seconds on the name of a desired destination - kitchen or bedroom, say - displayed on its screen. The computer then guides the wheelchair to the room using a preset programme.

For the Asimo demonstration, Honda used a combination of EEG and another non-invasive technique, Near Infrared Spectroscopy, which measures blood-flow changes in the brain. It's a real-time system, with a claimed accuracy of about 90 per cent.

Partially invasive BCIs, which come under the general term of electrocortigography (ECoG), offer advantages over EEG and implants but with fewer drawbacks. They measure signals with better resolution and signal-to-noise ratio than EEGs, and although, strictly speaking, they are still invasive, ECoG devices can be placed over areas of the brain that control speech and memory for example, where it would be risky to place implants.

ECoG-based BCIs are a more recent development - they were first used in humans in 2004 - but ECoG itself has been used for about 50 years to help treat people with severe epilepsy, and non-invasive alternatives are in the pipeline.

Microecog and epilepsy

One recent development here has been announced by University of Utah researchers, using a variant known as microECoG. In an experiment involving two people with severe epilepsy, the team's microECoG array successfully picked up brain signals so that the participants could accurately move their arms as directed.

Ultimately though, no matter which approach is used to detect brain activity, the signals still have to be analysed and interpreted. Detection is comparatively simple - doctors and scientists have been able to do it for about 80 years now - it's the signal analysis and interpretation that present the challenges.

It's impossible to mention here every strand of research in this field, but broadly the issue is one of applying algorithms to the signals. Techniques including autoregressive modelling, bandpass filtering, blind signal separation, fast fourier transforms, Gaussian mixture models and Kalman filters abound in the research literature, yet they are all looking to achieve essentially the same goal - to separate the brain recordings into their underlying constituent components and isolate the relevant source. And do it reliably, in real time.

It has been suggested that the best bits of the different algorithms could be combined into one generic version. But Southampton's Dr James, for one, is sceptical. "I am not really in favour of a generic BCI algorithm as I see it restricting growth in this area," he says. "BCI research is too 'young' to try to think of only one way of doing things yet; we need to keep looking outside the box right now."

Another important goal for practical devices is to enable intuitive and analogue thought control, something the BCI group at the University of Essex for example has been studying. The intuitive aspect is part of the group's research into adaptive asynchronous control, the advantages of which include replacing the traditionally unnatural user-interface with a more natural operating mode.

Wheelchair control

As the group's coordinator Dr Francisco Sepulveda explains, "If, for example, a wheelchair user wishes to turn the wheelchair to the left, it's easier to imagine a left-arm movement rather than some unrelated task such as mental arithmetic. This has been a challenge, however, as the most intuitive tasks are not always the easiest to classify automatically." But the aim of this research has now been achieved, he says, although improvements are still needed before the system can be used outside the laboratory.

An analogue system also holds promise, and the principle behind it is surprisingly simple. The group's Prof Riccardo Poli says, "The traditional wisdom in BCI is that brain-activity signals need to be processed and converted into digital commands so that a digital computer can interpret them. But to control a mouse pointer, say, a BCI needs to convert these digital commands back into a 2D motion of the pointer on the screen.

"This makes little sense - why convert analogue signals into digital then back to analogue? Our approach therefore is to try to exploit the natural amplitude variations in brain responses to stimuli to control mouse motion directly in an analogue fashion," he says. This is still in research at the moment, and results are not expected until 2011.

But results will come, as they will from BCI researchers the world over, and lead to products that will transform every aspect of all our lives - from disability to domesticity. And it will continue. Even now, for example, Dr James at Southampton is researching brain-to-brain communication. Now there's a thought.

Recent articles

Info Message

Our sites use cookies to support some functionality, and to collect anonymous user data.

Learn more about IET cookies and how to control them