Archives

For the first time scientists have used an implanted sensor to record brain signals associated with handwriting and used those signals to create text on a computer in real time.

As reported in the journal Nature, a clinical trial participant with cervical spinal cord injury used the system to “type” words on a computer at a rate of 90 characters per minute, more than double the previous record for typing with a brain-computer interface. The participant had to merely think about the hand motions involved in creating written letters to complete the task.

The research team is hopeful that such a system could one day help to restore people’s ability to communicate following paralysis caused by injury or illness.

The letters of the alphabet and then a "handwritten" version created using the brain-computer interface — A clinical trial participant created these letters on a computer screen just by thinking about the act of moving his arm and hand to write. (Credit: BrainGate.org)

The new study is part of the BrainGate clinical trial, directed by Leigh Hochberg, a critical care neurologist and a professor at Brown University’s School of Engineering affiliated with the Carney Institute for Brain Science.

Frank Willett, a research scientist at Stanford University and the Howard Hughes Medical Institute (HHMI), led the study, which was supervised by Krishna Shenoy, a Stanford professor and HHMI investigator, and Jaimie Henderson, a professor of neurosurgery at Stanford.

“An important mission of our BrainGate consortium research is to restore rapid, intuitive communication for people with severe speech or motor impairments,” says Hochberg, who also directs the Center for Neurotechnology and Neurorecovery at Massachusetts General Hospital and the VA Rehabilitation Research and Development Center for Neurorestoration and Neurotechnology at the Veterans Affairs Providence Healthcare System.

“Frank’s demonstration of fast, accurate neural decoding of handwriting marks an exciting new chapter in the development of clinically useful neurotechnologies.”

Faster communication

The BrainGate collaboration has worked for several years on systems that enable people to generate text through direct brain control. Previous incarnations have involved trial participants thinking about the motions involved in pointing to and clicking letters on a virtual keyboard. That system enabled one participant to type 40 characters per minute, which was the previous record speed.

For the new study, the team wanted to find out if asking a participant to think about motions involved in writing letters and words by hand would be faster.

“We want to find new ways of letting people communicate faster,” Willett says. “This new system uses both the rich neural activity recorded by intracortical electrodes and the power of language models that, when applied to the neurally decoded letters, can create rapid and accurate text.”

The trial participant, a 65-year-old (at the time of the study) man, was paralyzed from the neck down by a spinal cord injury. As part of the clinical trial, Henderson placed two tiny electrodes about the size of a baby aspirin in a part of his brain associated with the movement of his right arm and hand.

Using signals the sensors picked up from individual neurons when the man imagined writing, a machine learning algorithm recognized the patterns his brain produced with each letter. With this system, the man could copy sentences and answer questions at a rate similar to that of someone the same age typing on a smartphone.

The system is so fast because each letter elicits a highly distinctive activity pattern, making it relatively easy for the algorithm to distinguish one from another, Willett says.

Steady stream of BCI refinements

The new research is the latest in a series of advances in brain-computer interfaces (BCIs) made by the BrainGate collaboration, which includes researchers from Brown University, Massachusetts General Hospital, Harvard Medical School, the Providence VA Medical Center, Stanford University, and Case Western Reserve University.

In 2012, the team published landmark research in which clinical trial participants were able, for the first time, to operate multidimensional robotic prosthetics using a BCI.

A steady stream of refinements to the system, as well as new clinical breakthroughs that have enabled people to directly control tablet apps and even move their own paralyzed limbs, have followed that work. Most recently, the team demonstrated the first human use of a wireless intracortical BCI that can transmit neural data at full bandwidth.

Hochberg says he’s grateful to clinical trial participants for making these breakthroughs and future ones possible.

“The people who enroll in the BrainGate trial are amazing,” Hochberg says. “It’s their pioneering spirit that not only allows us to gain new insights into human brain function, but that leads to the creation of systems that will help other people with paralysis.”

The National Institute of Neurological Disorders and Stroke and the NIH BRAIN Initiative, the National Institute on Deafness and Other Communication Disorders, Howard Hughes Medical Institute, the US Department of Veterans Affairs, L. and P. Garlick, S. and B. Reeves, the Wu Tsai Neurosciences Institute at Stanford, and the Simons Foundation Collaboration on the Global Brain funded the work.

Source: Brown University

A brain-to-computer hookup recently allowed people with severe limb weakness to type via direct brain control at the highest speeds and accuracy levels reported to date.

Two of the participants have amyotrophic lateral sclerosis, also called Lou Gehrig’s disease, and one has a spinal cord injury.

They each had one or two baby-aspirin-sized electrode arrays placed in their brains to record signals from the motor cortex, a region controlling muscle movement. The signals were transmitted to a computer via a cable and translated by algorithms into point-and-click commands guiding a cursor to characters on an onscreen keyboard.

Each participant, after minimal training, mastered the technique sufficiently to outperform the results of any previous test of brain-computer interfaces, or BCIs, for enhancing communication by people with similarly impaired movement. Notably, they achieved the typing rates without the use of automatic word-completion assistance common in electronic keyboarding applications nowadays, which likely would have boosted their performance.

One participant, Dennis Degray of Menlo Park, California, was able to type 39 correct characters per minute, equivalent to about eight words per minute.

This point-and-click approach could be applied to a variety of computing devices, including smartphones and tablets, without substantial modifications, the researchers say. Their findings appear in the journal eLife.

“This is like one of the coolest video games I’ve ever gotten to play with. And I don’t even have to put a quarter in it.”

“Our study’s success marks a major milestone on the road to improving quality of life for people with paralysis,” says Jaimie Henderson, professor of neurosurgery at Stanford University, who performed two of the three device-implantation procedures at Stanford Hospital. The third took place at Massachusetts General Hospital.

“This study reports the highest speed and accuracy, by a factor of three, over what’s been shown before,” says co-senior author Krishna Shenoy, professor of electrical engineering. “We’re approaching the speed at which you can type text on your cellphone.”

“The performance is really exciting,” says former postdoctoral scholar Chethan Pandarinath, who now has a joint appointment at Emory University and the Georgia Institute of Technology as an assistant professor of biomedical engineering. “We’re achieving communication rates that many people with arm and hand paralysis would find useful. That’s a critical step for making devices that could be suitable for real-world use.”

Shenoy’s lab pioneered the algorithms used to decode the complex volleys of electrical signals fired by nerve cells in the motor cortex, the brain’s command center for movement, and convert them in real time into actions ordinarily executed by spinal cord and muscles.

“These high-performing BCI algorithms’ use in human clinical trials demonstrates the potential for this class of technology to restore communication to people with paralysis,” says postdoctoral scholar Paul Nuyujukian.

‘I was taking out the trash in the rain’

Millions of people with paralysis live in the United States. Sometimes their paralysis comes gradually, as occurs in ALS. Sometimes it arrives suddenly, as in Degray’s case.

Now 64, Degray became quadriplegic on October 10, 2007, when he fell and sustained a life-changing spinal-cord injury. “I was taking out the trash in the rain,” he said. Holding the garbage in one hand and the recycling in the other, he slipped on the grass and landed on his chin. The impact spared his brain but severely injured his spine, cutting off all communication between his brain and musculature from the head down. “I’ve got nothing going on below the collarbones,” he says.

Watch paralyzed man move robotic arm with his mind

Degray received two device implants at Henderson’s hands in August 2016. In several ensuing research sessions, he and the other two study participants, who underwent similar surgeries, were encouraged to attempt or visualize patterns of desired arm, hand, and finger movements. Resulting neural signals from the motor cortex were electronically extracted by the embedded recording devices, transmitted to a computer and translated by Shenoy’s algorithms into commands directing a cursor on an onscreen keyboard to participant-specified characters.

The quick brown fox…

The researchers gauged the speeds at which the patients were able to correctly copy phrases and sentences—for example, “The quick brown fox jumped over the lazy dog.” Average rates were 7.8 words per minute for Degray and 6.3 and 2.7 words per minute, respectively, for the other two participants.

Rerouted nerves let paralyzed people use hands

The investigational system used in the study, an intracortical brain-computer interface called the BrainGate Neural Interface System, represents the newest generation of BCIs. Previous generations picked up signals first via electrical leads placed on the scalp, then by being surgically positioned at the brain’s surface beneath the skull.

An intracortical BCI uses a tiny silicon chip, just over one-sixth of an inch square, from which protrude 100 electrodes that penetrate the brain to about the thickness of a quarter and tap into the electrical activity of individual nerve cells in the motor cortex.

Henderson likened the resulting improved resolution of neural sensing, compared with that of older-generation BCIs, to that of handing out applause meters to individual members of a studio audience rather than just stationing them on the ceiling, “so you can tell just how hard and how fast each person in the audience is clapping.”

24/7 wireless system

The day will come—closer to five than 10 years from now, Shenoy predicts—when a self-calibrating, fully implanted wireless system can be used without caregiver assistance, has no cosmetic impact. and can be used around the clock.

“I don’t see any insurmountable challenges,” he says. “We know the steps we have to take to get there.”

Degray, who continues to participate actively in the research, knew how to type before his accident but was no expert at it. He described his newly revealed prowess in the language of a video game aficionado.

“This is like one of the coolest video games I’ve ever gotten to play with,” he says. “And I don’t even have to put a quarter in it.”

Stanford research assistant Christine Blabe is also a study coauthor, as are BrainGate researchers from Massachusetts General Hospital and Case Western University.

Funding came from the National Institutes of Health, the Stanford Office of Postdoctoral Affairs, the Craig H. Neilsen Foundation, the Stanford Medical Scientist Training Program, Stanford BioX-NeuroVentures, the Stanford Institute for Neuro-Innovation and Translational Neuroscience, the Stanford Neuroscience Institute, Larry and Pamela Garlick, Samuel and Betsy Reeves, the Howard Hughes Medical Institute, the US Department of Veterans Affairs, the MGH-Dean Institute for Integrated Research on Atrial Fibrillation and Stroke, and Massachusetts General Hospital.

Stanford’s Office of Technology Licensing holds intellectual property on the intercortical BCI-related engineering advances made in Shenoy’s lab.

Source: Bruce Goldman for Stanford University