A vest that allows the profoundly deaf to “feel” and understand speech is under development.
The vest features dozens of embedded sensors that vibrate in specific patterns to represent words. VEST—Versatile Extra-Sensory Transducer—responds to input from a phone or tablet app that isolates speech from ambient sound.
A team of undergraduates is working on VEST with Scott Novich, a doctoral student in electrical and computer engineering at Rice University who works in the lab of neuroscientist David Eagleman.
Novich devised the algorithm that enables the VEST to “hear” only the human voice and screen out distracting sounds.
‘Feeling the sonic world’
The low-cost, noninvasive vest collects sounds from a mobile app and converts them into tactile vibration patterns on the user’s torso. Haptic feedback supplants auditory input.
The first VEST prototype put together by the team has 24 actuators sewn into the back. A second version, already in production, will include 40 of the actuators Eagleman calls “vibratory motors.” He described the experience, at least for a hearing person, as “feeling the sonic world around me.”
“Along with all the actuators, the system includes a controller board and two batteries,” says Gary Woods, the team’s adviser and a professor in the practice of computer technology. “The actuators vibrate in a very complicated pattern based on audio fed through a smartphone. The patterns are too complicated to translate consciously.”
With training, the brains of deaf people adapt to the “translation” process, Eagleman says. Test subjects, some of them deaf from birth, “listened” to spoken words and wrote them on a white board. “They can start understanding the ‘language’ of the vest,” he says.
New words, new skills
“We’ve already run some simple experiments with both hearing and deaf people,” Novich says. “As they use the vest more, they get feedback, and know whether they are right or wrong and start to memorize patterns. People are able to identify words they have never encountered before.”
[related]
The project has also prompted the students to learn skills they wouldn’t necessarily acquire in engineering classrooms. Senior Zihe Huang became the team’s tailor when he learned to sew via YouTube. “I’m an electrical engineer,” he says. “I didn’t know anything about sewing.”
The team will present its work this month at the annual Design of Medical Devices conference in Minnesota.
“We see other applications for what we’re calling tactile sensory substitution,” says junior Abhipray Sahoo. “Information can be sent through the human body. It’s not just an augmentative device for the deaf. The VEST could be a general neural input device. You could receive any form of information.”
The project is a collaboration with Baylor College of Medicine.
Source: Rice University
