Using a noninvasive brain-computer interface, researchers have developed the first-ever successful mind-controlled robotic arm with the ability to continuously track and follow a computer cursor.

Being able to noninvasively control robotic devices using only thoughts will have broad applications, in particular benefiting the lives of paralyzed patients and those with movement disorders.

Brain-computer interfaces (BCIs) have been shown to achieve good performance for controlling robotic devices using only the signals sensed from brain implants. When robotic devices can be controlled with high precision, they can complete a variety of daily tasks.

Until now, however, BCIs successful in continuously controlling robotic arms have used invasive brain implants. These implants require a substantial amount of medical and surgical expertise to correctly install and operate, not to mention cost and potential risks to subjects. As such, their use has been limited to just a few clinical cases.

‘Mind control’

A grand challenge in BCI research is to develop less invasive or even totally noninvasive technology that would allow paralyzed patients to control their environment or robotic limbs using their own “thoughts.” Such noninvasive BCI technology, if successful, would bring such much-needed technology to numerous patients and even potentially to the general population.

However, BCIs that use noninvasive external sensing, rather than brain implants, receive “dirtier” signals, leading to lower resolution and less precise control. Thus, when using only the brain to control a robotic arm, a noninvasive BCI doesn’t stand up to using implanted devices. Despite this, BCI researchers have forged ahead, their eye on the prize of a less- or non-invasive technology that could help patients everywhere on a daily basis.

“There have been major advances in mind controlled robotic devices using brain implants. It’s excellent science,” says Bin He, department head and professor of biomedical engineering at Carnegie Mellon University. “But noninvasive is the ultimate goal. Advances in neural decoding and the practical utility of noninvasive robotic arm control will have major implications on the eventual development of noninvasive neurorobotics.”

Using novel sensing and machine learning techniques, He and his lab have been able to access signals deep within the brain, achieving a high resolution of control over a robotic arm. With noninvasive neuroimaging and a novel continuous pursuit paradigm, He is overcoming the noisy EEG signals leading to significantly improve EEG-based neural decoding, and facilitating real-time continuous 2D robotic device control.

Using a noninvasive BCI to control a robotic arm that’s tracking a cursor on a computer screen, for the first time ever, He has shown in human subjects that a robotic arm can now follow the cursor continuously. Whereas robotic arms controlled by humans noninvasively had previously followed a moving cursor in jerky, discrete motions—as though the robotic arm was trying to “catch up” to the brain’s commands—now, the arm follows the cursor in a smooth, continuous path.

Better brain signals

In a paper in Science Robotics, the team establishes a new framework that addresses and improves upon the “brain” and “computer” components of BCI by increasing user engagement and training, as well as spatial resolution of noninvasive neural data through EEG source imaging.

The paper shows that the team’s unique approach to solving this problem not only enhanced BCI learning by nearly 60 percent for traditional center-out tasks, it also enhanced continuous tracking of a computer cursor by more than 500 percent.

The technology also has applications that could help a variety of people, by offering safe, noninvasive “mind control” of devices that can allow people to interact with and control their environments. The technology has, to date, been tested in 68 able-bodied human subjects (up to 10 sessions for each subject), including virtual device control and controlling of a robotic arm for continuous pursuit. The technology is directly applicable to patients, and the team plans to conduct clinical trials in the near future.

“Despite technical challenges using noninvasive signals, we are fully committed to bringing this safe and economic technology to people who can benefit from it,” says He. “This work represents an important step in noninvasive brain-computer interfaces, a technology which someday may become a pervasive assistive technology aiding everyone, like smartphones.”

The National Center for Complementary and Integrative Health, National Institute of Neurological Disorders and Stroke, National Institute of Biomedical Imaging and Bioengineering, and National Institute of Mental Health supported the work, in part.

Source: Carnegie Mellon University

Related

Robot copies Mona Lisa sketch just by looking at it

App and suitcase aid visually impaired people at the airport

These robot arms can fold laundry

A new low-cost, human-friendly robot uses recent advances in artificial intelligence and deep reinforcement learning to master intricate human tasks.

Robots may have a knack for super-human strength and precision, but they still struggle with some basic human tasks—like folding laundry or making a cup of coffee.

That’s where the new robot, named Blue, comes in. Researchers have designed Blue to be capable while remaining affordable and safe enough that every artificial intelligence researcher—and eventually every home—could have one.

Blue is the brainchild of Pieter Abbeel, a professor of electrical engineering and computer sciences at the University of California, Berkeley; postdoctoral research fellow Stephen McKinley; and graduate student David Gealy. The team hopes Blue will accelerate the development of robotics for the home.

A robot for the AI moment

“AI has done a lot for existing robots, but we wanted to design a robot that is right for AI,” Abbeel says.

“Existing robots are too expensive, not safe around humans, and similarly not safe around themselves—if they learn through trial and error, they will easily break themselves. We wanted to create a new robot that is right for the AI age rather than for the high-precision, sub-millimeter, factory automation age.”

“We wanted to design the weakest robot that could still do really useful stuff.”

Over the past 10 years, Abbeel has pioneered deep reinforcement learning algorithms that help robots learn by trial and error or through human guidance like a puppet. He developed these algorithms using robots from outside companies, which market them for tens of thousands of dollars.

Blue’s durable, plastic parts and high-performance motors total less than $5,000 to manufacture and assemble. Its arms, each about the size of the average bodybuilder’s, are sensitive to outside forces—like a hand pushing it away—and has rounded edges and minimal pinch points to avoid catching stray fingers. Blue’s arms can be very stiff, like a human flexing, or very flexible, like a human relaxing, or anything in between.

“With a lower-cost robot, every researcher could have their own robot, and that vision is one of the main driving forces behind this project—getting more research done by having more robots in the world,” McKinley says.

Going with the flow

Robotics has traditionally focused on industrial applications, where robots need strength and precision to carry out repetitive tasks perfectly every time. These robots flourish in highly structured, predictable environments—a far cry from the traditional American home, where you might find children, pets, and dirty laundry on the floor.

“We’ve often described these industrial robots as moving statues,” Gealy says. “They are very rigid, meant to go from point A to point B and back to point A perfectly. But if you command them to go a centimeter past a table or a wall, they are going to smash into the wall and lock up, break themselves, or break the wall. Nothing good.”

If an AI is going to make mistakes and learn by doing in unstructured environments, these rigid robots just won’t work. To make experimentation safer, Blue was designed to be force-controlled—highly sensitive to outside forces, always modulating the amount of force it exerts at any given time.

“One of the things that’s really cool about the design of this robot is that we can make it force-sensitive, nice and reactive, or we can choose to have it be very strong and very rigid,” Gealy says.

“Researchers can adjust how stiff the robot is, and what kind of stiffness—do you want it to feel like molasses? Do you want it to feel like a spring? A combination of those? If we want robots to move toward the home and perform in these increasingly unstructured environments, they are going to need that capability.”

To achieve these capabilities at low cost, the team considered what features Blue needed to complete human-centered tasks, and what it could go without. For example, the researchers gave Blue a wide range of motion—it has joints that can move in the same directions as a human shoulder, elbow, and wrist—to enable humans to more easily teach it how to complete tricky maneuvers using virtual reality. But the agile robot arms lack some of the strength and precision of a typical robot.

“What we realized was that you don’t need a robot that exerts a specific force for all time, or a specific accuracy for all time. With a little intelligence, you can relax those requirements and allow the robot to behave more like a human being to achieve those tasks,” McKinley says.

Weaker bot, more power

Blue is able to continually hold up 2 kilograms [just under 4.5 pounds] of weight with arms fully extended. But unlike traditional robot designs that are characterized by one consistent “force/current limit,” Blue is designed to be “thermally-limited,” McKinley says.

That means that, similar to a human being, it can exert a force well beyond 2 kilograms in a quick burst, until it reaches its thermal limits and it needs time to rest or cool down. This is just like how a human can pick up a laundry basket and easily carry it across a room, but might not be able to carry the same laundry basket over a mile without frequent breaks.

“Essentially, we can get more out of a weaker robot,” Gealy says. “And a weaker robot is just safer. The strongest robot is most dangerous. We wanted to design the weakest robot that could still do really useful stuff.”

“Researchers had been developing AI for existing hardware and, about three years ago, we began thinking, ‘Maybe we could do something the other way around. Maybe we could think about what hardware we could build to augment AI and work on those two paths together, at the same time,'” McKinley says. “And I think that is a really dramatic shift from the way a lot of research has taken place.”

Currently, the team is building 10 arms in-house to distribute to select early adopters. They are continuing to investigate Blue’s durability and to tackle the formidable challenge of manufacturing the robot on a larger scale, which will happen through the UC Berkeley spinoff Berkeley Open Arms, where sign-ups for expressing interest in priority access are now open.

Source: UC Berkeley

Related

What are the dangers and downsides of A.I.?

Test yourself: Can you think like a computer?

Is artificial intelligence actually ‘kind of dumb’?