‘PrivacyMic’: the smart speaker that doesn’t eavesdrop

Microphones are among the most common electronic sensors in the world, with about 320 million listening for our commands in the world’s smart speakers. However, such technology can hear everything else, too.

To challenge this, the ‘PrivacyMic’ device, developed by a team at the University of Michigan (U-M), uses ultrasonic sound at frequencies above the range of human hearing. Running dishwashers, computer monitors, even finger snaps, all generate ultrasonic sounds, which have a frequency of 20kHz or higher.

The system pieces together the ultrasonic information in its environment to identify when a user needs its services, and sense what’s going on around it. The researchers have showed that it can identify household and office activities with greater than 95 per cent accuracy.

“There are a lot of situations where we want our home automation system or our smart speaker to understand what’s going on in our home, but we don’t want it listening to our conversations,” said Alanson Sample, U-M associate professor of electrical engineering and computer science. “And what we've found is that you can have a system that understands what’s going on and a hard guarantee that it will never record any audible information.”

The researchers said PrivacyMic can filter out audible information right on the device, making it more secure than encryption or other security measures that take steps to secure audio data after it’s recorded or limit who has access to it. Those measures could all leave sensitive information vulnerable to hackers, but with PrivacyMic, the information simply doesn’t exist, the team said.

The idea behind PrivacyMic began when the team was classifying previously recorded audio. Looking at a visual graph of the data, the researcher found that audible sound was only a small piece of what was available.

“We realised we were sitting on a lot of interesting information that was being ignored. We could actually get a picture of what was going on in a home or office without using any audio at all,” said Yasha Iravantchi, a graduate student in electrical engineering and computer science

With this knowledge, a laptop, and an ultrasonic microphone, the team then captured audio from tooth brushing, toilet flushing, vacuuming, running dishwashers, using computer monitors, and hundreds of other common activities. They then compressed the ultrasonic signatures into smaller files that included key bits of information while stripping out noise within the range of human hearing – similar to an ultrasonic MP3 – and built a Raspberry Pi-based device to listen for them.

According to the team, the device, which can be set to filter out speech or to strip out all audible content, accurately identified common activities over 95 per cent of the time. The team also conducted a trial where study participants listened to the audio collected by the device and found that not a single participant could make out human speech.

While the device is a proof of concept at this stage, Sample says that implementing similar technology in a device like a smart speaker would require only minor modifications – software that listens for ultrasonic sound and a microphone capable of picking it up, which are inexpensive and readily available. 

“Smart technology today is an all-or-nothing proposition. You can either have nothing or you can have a device that’s capable of constant audio recording,” Sample said. “PrivacyMic offers another layer of privacy – you can interact with your device using audio if you choose or you can have another setting where the device can glean information without picking up audio.”