IE 11 is not supported. For an optimal experience visit our site on another browser.

Thumbs Up: Mind-Controlled Robot Arm Wins at Rock-Paper-Scissors

A paralyzed woman ends a two-year project learning to control a robotic arm with thought alone by beating her brother at a playground favorite.
A robot hand at the University of Pittsburgh Medical Center offers hope to those with spinal injuries.
A robot hand at the University of Pittsburgh Medical Center offers hope to those with spinal injuries. University of Pittsburgh Medical Center

A paralyzed woman who was one of the first people to control a robotic arm using thought alone has finished a two-year run on a high note — by beating her brother at rock-paper-scissors.

Jan Scheuermann was the first patient to try out a new mind-controlled arm being developed at the University of Pittsburgh. Scheuermann, 55, is paralyzed from the neck down by a degenerative condition. She made news at the end of 2012 when she fed herself a bar of chocolate using the arm.

Two years later, the medical team say they’ve fine-tuned the often clumsy movements of the device, which Scheuermann nicknamed “Hector.”

“It just looked like a ‘Hector,’” explained Scheuermann, who pronounces her last name like “Sherman” and who spent two years with bolts from neural implants sticking out of the top of her skull. Those she nicknamed “Lewis” and “Clark.” “That’s because they’re exploring and charting new territory,” she told NBC News.

“We later found out that, in Greek, ‘Hector’ means ‘to grasp.’ How appropriate. That was so cool.”

“Highly coordinated, natural arm and hand movement can be restored to people with upper limb paralysis.”

The implants have been taken out now, and she’s back to a normal, quiet life without hours in the lab.

At the end, Scheuermann could move “Hector” in seven directions, do a thumbs-up, pick up fiddly objects and finesse the playground favorite, rock-paper-scissors. “This study demonstrates, for the first time, brain-machine interface control of an anthropomorphic prosthetic arm that includes continuous control of multiple dimensions of hand shape,” the researchers write in Wednesday’s issue of the Journal of Neural Engineering.

The results show that “highly coordinated, natural arm and hand movement can be restored to people with upper limb paralysis.”

Other people have tried out mind-controlled limbs. Several are in people who have lost limbs, and they connect to living nerves. Hector is controlled by thought alone, with no physical connection to the body. It’s a little like the arm controlled by monkeys in Miguel Nicolelis’s lab at Duke University. His team has also made mind-controlled exoskeletons, most dramatically used to help a paralyzed man kick off the World Cup soccer tournament.

Jan Scheuermann uses Hector, a robotic arm, to play "rock, paper, scissors" with her brother Ed Scheuermann.
Jan Scheuermann uses Hector, a robotic arm, to play "rock, paper, scissors" with her brother Ed Scheuermann. Courtesy Jan Scheuermann

What Scheuermann and the University of Pittsburgh researchers would like to see some day is a small, portable robotic arm. "The hope is that the technology will be wireless and it can be mounted on someone’s wheelchair,” Scheuermann said. She’d be the first in line to get one.

“They said before I started, ‘You will not be able to take him home at any time,'" she said. “I knew that someone else would come along and do it after me.”

Even so, she said, she “bawled” the night after surgeons removed the neural arrays that sent her brain’s signals to the computer. “I was racked with sobs for, like, 15 minutes,” she said. “Then I wiped my eyes.”

Dr. Jennifer Collinger, one of the researchers who’s been working on the project, says that’s one of the sensitivities of working with real people.

“She was only ever able to use that in the lab and I think she understood that from the beginning,” Collinger told NBC News. “It takes a special kind of person to know that from the beginning and still undergo surgery.”

Scheuermann has contributed more than her fair share of time for now, Collinger said. “She committed two and a half years to our study. I think that is probably enough until we get to the stage of her potentially using it in her home and having a benefit,” Collinger added.

Now the team will fine-tune the device and see if they can make it usable in day-to-day life. Collinger is hoping in five to 10 years, the team can come up with a workable product.

“We are still looking at her data to learn a lot of things,” she said. For one thing, Scheuermann was only able to get Hector to make certain maneuvers if she closed her eyes.

“This has been a huge blessing in my life.”

And sometimes the hand worked well on its own and then stalled when an object was there. The researchers are not sure why, and they need to know what changed in Scheuermann’s brain when she was faced with handling an actual object in certain ways.

“When you are interacting with objects, it seems to change the way your motor cortex responds,” Collinger said.

The project is funded by the Defense Advanced Research Projects Agency, or DARPA, with wounded veterans and other patients in mind. The device cost several hundred thousand dollars but with more production the price would come down, the researchers say. They'll be recruiting more volunteers.

Scheuermann, who was diagnosed in 1998 and who was almost completely paralyzed by 2003, says she can wait. “I did this first and no one can take that away from me,” she said.

“This has been a huge blessing in my life.”