Laser beams can trick a voice-controlled virtual assistants like Siri, Alexa, or Google Assistant into acting as if they registered an audio command, researchers report.
This trick worked at a distance of more than 300 feet and through a glass window.
The researchers discovered in the microphones of these systems a vulnerability that they call “Light Commands.” They also propose hardware and software fixes, and they’re working with Google, Apple, and Amazon to put them in place.
“We’ve shown that hijacking voice assistants only requires line-of-sight rather than being near the device,” says Daniel Genkin, assistant professor of computer science and engineering at the University of Michigan. “The risks associated with these attacks range from benign to frightening depending on how much a user has tied to their assistant.
“In the worst cases, this could mean dangerous access to homes, e-commerce accounts, credit cards, and even any connected medical devices the user has linked to their assistant.”
The team showed that Light Commands could enable an attacker to remotely inject inaudible and invisible commands into smart speakers, tablets, and phones in order to:
Unlock a smart lock-protected front door
Open a connected garage door
Shop on e-commerce websites at the target’s expense
Locate, unlock, and start a vehicle that’s connected to a target’s account
Just five milliwatts of laser power—the equivalent of a laser pointer—was enough to obtain full control over many popular Alexa and Google smart home devices, while about 60 milliwatts was sufficient in phones and tablets.
To document the vulnerability, the researchers aimed and focused their light commands with a telescope, a telephoto lens, and a tripod. They tested 17 different devices representing a range of the most popular assistants.
“There is a semantic gap between what the sensors in these devices are advertised to do and what they actually sense, leading to security risks,” says Kevin Fu, an associate professor of computer science and engineering . “In Light Commands, we show how a microphone can unwittingly listen to light as if it were sound.”
Users can take some measures to protect themselves from Light Commands.
“One suggestion is to simply avoid putting smart speakers near windows, or otherwise attacker-visible places,” says Sara Rampazzi, a postdoctoral researcher in computer science and engineering. “While this is not always possible, it will certainly make the attacker’s window of opportunity smaller. Another option is to turn on user personalization, which will require the attacker to match some features of the owner’s voice in order to successfully inject the command.”
Additional researchers from the University of Electro-Communications in Tokyo and the University of Michigan contributed to the work.
Researchers have shown how it’s possible to transform a smart device into a surveillance tool that can collect information about the body position and movements of people near the device.
Their approach involves remotely hijacking smart devices to play music embedded with repeating pulses that track a person’s position, body movements, and activities, both in the vicinity of the device as well as through walls.
As smartphones, tablets, smart TVs, and other smart devices become more prevalent in our lives, computer scientists have raised concerns that these network-enabled devices, if not properly secured, could be co-opted to steal data or invade user privacy.
The researchers tested CovertBand using a Samsung Galaxy S4 smartphone hooked up to a portable speaker. (Credit: Dennis Wise/U. Washington)
The team showed how it is possible to collect such detailed data on personal activity using CovertBand, software code they created to turn smart devices into active sonar systems. CovertBand can utilize built-in microphones and speakers in a smart device—and can be controlled remotely.
From smart device to sonar system
“To our knowledge, this is the first time anyone has demonstrated that it is possible to convert smart commodity devices—like smart phones and smart TVs—into active sonar systems using music,” says senior author Shyam Gollakota, an associate professor of computer science and engineering.
“And the physical information CovertBand can gather—even through walls—is sufficiently detailed for an attacker to know what the user is doing, as well as other people nearby,” he says.
“…these devices have the basic components in place to make them vulnerable to attack…”
CovertBand utilizes the principles of active sonar to gather this information. Active sonar systems, such as on submarines, determine the position of objects by sending out an acoustic pulse. Those sound waves bounce off objects in their path, and the deflected waves can be picked up by a receiver to determine the object’s position, distance, and shape.
Through the speaker of a smartphone or other device, CovertBand sends out a repeating pulse of sound waves in the 18 to 20 kHz range. Much like sonar on a submarine, these sound waves are reflected when they encounter objects in their path. CovertBand uses the device’s built-in microphones as a receiver to pick up these reflected sound waves. The smart device then transmits this information to the attacker, who could be a few feet away or halfway across the globe.
The screen shows the signatures of arm waving, as detected by CovertBand. CovertBand can remotely transform a smart device into an active sonar system, using its microphones to transmit a repeated audio pulse and its speakers to collect spatial information on how those pulses are reflected due to the repetitive motions of users. (Credit: Dennis Wise/U. Washington)
“Most of today’s smart devices including smart TVs, Google Home, Amazon Echo, and smartphones come with built-in microphones and speaker systems—which lets us use them to play music, record video and audio tracks, have phone conversations, or participate in videoconferencing,” says co-lead author Rajalakshmi Nandakumar, a doctoral student in computer science and engineering at the University of Washington. “But that also means that these devices have the basic components in place to make them vulnerable to attack in this manner.”
“Other surveillance approaches require specialized hardware, from the ‘classic’ hidden camera to an ultrasound-like device that must be placed on the wall of a neighboring room,” says co-lead author Alex Takakuwa, a doctoral student in computer science and engineering. “CovertBand shows for the first time that through-barrier surveillance is possible using no hardware beyond what smart devices already have.”
Testing, testing
The team tested CovertBand’s effectiveness using a smartphone hooked up to either a portable speaker or a standard flat-screen TV. In both cases, CovertBand’s data could be used to decipher repetitive movements such as arm-pumping, walking, or pelvic tilts to a range of up to 6 meters from the smartphone, with a positional error of only 8 to 18 centimeters. Researchers also discovered that, with the portable speaker, CovertBand’s pulses can transmit through thin, interior walls—though the range drops to 2 to 3 meters.
Currently, CovertBand can automatically identify and infer repetitive motions. More detailed inferences require manual analyses of data—or additional tools.
“Our initial goal was to demonstrate that it is possible to use passive acoustics to gather even basic—but still highly sensitive—information using CovertBand,” says Gollakota. “But if you have enough data from CovertBand, you could run it through machine-learning algorithms to help classify more movements for faster identification.”
“…if a neighbor is playing music, it could either be a benign act or an act of surveillance…”
The 18 to 20 kHz repeating pulses employed by CovertBand are on the low range of what many people can hear accurately, though children, younger adults, and even pets might be able to hear it well, says Nandakumar. But to increase the range of surveillance and work through walls, the authors increased the volume of these repeating pulses, which made them audible.
To mask the sound, they “covered” Covertband’s pulses by playing songs or other audio clips over them. Some songs work better than others—particularly compositions with repetitive, percussive beats. When they played the CovertBand pulses beneath 20 popular songs—including Lenny Kravitz’s “American Woman” and Michael Jackson’s “Bad”—listeners could identify the “hacked” version of the song 58 percent of the time, just slightly above the 50 percent accuracy expected by guessing randomly.
Protection and defenses
“Since Covertband enables through-the-wall surveillance, anyone can play music on their smart devices to track people through walls,” says Takakuwa. “This is concerning because, if a neighbor is playing music, it could either be a benign act or an act of surveillance to determine if anyone is in the adjacent apartment, track their movements, or infer their activities.”
The researchers say that soundproofing a room would prevent attacks through walls. Emitting a jamming signal at the same 18 to 20 kHz frequency range would also prevent hacked devices or attackers in the next room from gathering information. But currently, those are also impractical defenses for most people. Soundproofed rooms have no windows, for example, and jamming signals would have to be sent the moment an attack is detected.
Another potential—though partial—defense could be to allow users to deactivate the speakers or microphones on their smart devices. But such a move would go against industry trends for some of these devices.
“In many cases, when the device is on, then its speakers and microphones are also on,” says Nandakumar.
The team hopes that knowledge of what is possible will help develop awareness of privacy dangers and prompt scientists to develop practical countermeasures.
“We always want to stay one step ahead of the bad guys—of attackers who are trying to collect this information about users,” says coauthor Tadayoshi Kohno, a professor of computer science and engineering. “We’re providing education about what is possible and what capabilities the general public might not know about, so that people can be aware and can build defenses against this.”
A Google Faculty Award, the Alfred P. Sloan Foundation, the University of Washington’s Short-Dooley Career Development Professorship, and the National Science Foundation funded the research in part.
