Sensors able to detect phone vibrations to ‘eavesdrop remotely’

Using an off-the-shelf automotive radar sensor and a novel processing approach, a team of scientists has been able to "eavesdrop remotely" on other people's phone conversations. 

The scientists demonstrated their technology in the 2022 Institute of Electrical and Electronics Engineers Symposium on Security and Privacy, with a view to warning against this possible security flaw.

“As technology becomes more reliable and robust over time, the misuse of such sensing technologies by adversaries becomes probable,” doctoral candidate Suryoday Basak said.

“Our demonstration of this kind of exploitation contributes to the pool of scientific literature that broadly says, ‘Hey! Automotive radars can be used to eavesdrop audio. We need to do something about this.’”

In the ‘mmSpy’ demonstration, the researchers simulated people speaking through the earpiece of a smartphone. The phone’s earpiece vibrated from the speech and that vibration permeated across the body of the phone and was picked up by the sensor, even when the audio was completely inaudible to both humans and microphones nearby.

The results were the same irrespective of the brand of phone used, Basak said. The team tested their approach on both a Google Pixel 4a and a Samsung Galaxy S20.

The radar used by the team operated in the millimetre-wave (mmWave) spectrum, specifically in the bands of 60-64 gigahertz and 77-81 gigahertz, which inspired the researchers to name their approach ‘mmSpy’. This is a subset of the radio spectrum used for 5G, the fifth-generation standard for communication systems across the globe. 

“We use the radar to sense this vibration and reconstruct what was said by the person on the other side of the line,” Basak said. “This isn't the first time similar vulnerabilities or attack modalities have been found, but this particular aspect - detecting and reconstructing speech from the other side of a smartphone line - was not yet explored.” 

The team pre-processed the radar sensor data via MATLAB and Python modules and fed it into machine-learning modules trained to classify speech and reconstruct audio.

The findings of the team showed that the sensor could detect speech with 83 per cent accuracy when placed a foot away from the phone. The accuracy dropped the farther the radar moved from the phone, down to 43 per cent accuracy at six feet. 

Once the speech is reconstructed, the researchers can then filter, enhance or classify keywords as needed, Basak said.

Currently, the team is continuing to refine their approach to better understand not only how to protect against this security vulnerability, but also how to exploit it for good. 

“The methodology that we developed can also be used for sensing vibrations in industrial machinery, smart home systems and building-monitoring systems,” Basak said. “Vibration tracking over time can help assess wear and tear; using our approach could help identify when machinery needs maintenance before it would traditionally be obvious, for example.

“With the right set of target actions, radars in smart homes and industry can enable a faster turnaround when problems and issues are detected.”