Eye tracking system determines drivers’ ability to take back control from autopilot

The research found that people’s attention levels and how engrossed they are in on-screen activities can be detected from their eye movements.

Although fully autonomous driverless cars are not yet available for personal use, cars with a ‘driverless’ autopilot mode are available for commercial private use in some locations, including Germany and certain US states.

Tesla, for example, has an ‘Autopilot’ mode, which can steer, accelerate and brake within lanes, while ‘Full Self-Driving’ lets vehicles obey traffic signals and change lanes. But both technologies “require active driver supervision”, with a “fully attentive” driver whose hands are on the wheel, “and do not make the vehicle autonomous”.

Drivers can, for example, use the limited driverless functionality during a traffic jam on a motorway – but once the jam has cleared and the motorway allows faster than 40mph speeds, the AI will send a ‘takeover’ signal to the driver, indicating that they must return to full driving control.

The researchers tested whether it was possible to detect if a person was too engrossed in another task to respond swiftly to such a ‘takeover’ signal.

To do this, the team tested 42 participants across two experiments, using a procedure that mimicked a ‘takeover’ scenario as used in some advanced models of cars with an autopilot mode.

Participants were required to search a computer screen with many coloured shapes for some target items and linger their gaze on targets to show they had found them.

The easiest search tasks saw participants having to spot an odd ‘L’ shape amongst multiple ‘T’ shapes, while the more demanding tasks required participants to spot a specific arrangement of the shape parts and their colour.

At later points in their search task, a tone would then sound and the participants were required to stop watching the screen as fast as they could and press a button in response to it.

Researchers monitored the time it took between the tone sounding and the participants pressing the button, alongside analysing how their eyes moved across the screen during their search, to see if attention levels to the task could be detected from a change in their gaze.

They found that when the task demanded more attention, participants took a longer time to stop watching the screen and respond to the tone.

The analysis showed that it was possible to detect participants’ attention levels from their eye movements. An eye movement pattern involving longer fixations and shorter distance of eye travel between all items indicated that the task was more demanding on attention.

The researchers also trained a machine learning model on this data and found that they could predict whether the participants were engaged in the easy or demanding task based on their eye movement patterns.

Professor Nilli Lavie, senior author of the study, said: “Driverless car technology is fast advancing and promises a more enjoyable and productive driving experience, where drivers can use their commuting time for other non-driving tasks.

“However, the big question is whether the driver will be able to return to driving swiftly upon receiving a takeover signal if they are fully engaged in another activity.

“Our findings show that it is possible to detect the attention levels of a driver and their readiness to respond to a warning signal just from monitoring their gaze pattern.

“It is striking that people can get so consumed with their on-screen activity that they ignore the rest of the world around them. Even when they are aware that they should be ready to stop their task and respond to tones as quickly as they can, they take longer to do it when their attention is engrossed in the screen.

“Our research shows that warning signals may not be noticed quickly enough in such cases.”

Larger datasets are required in order to train the machine learning and make it more accurate, the researchers said.