Intuitive control of robotic arm using thoughts alone

Through a clinical collaboration between Caltech, Keck Medicine of USC and Rancho Los Amigos National Rehabilitation Center, the now 34-year-old Sorto is the first person in the world to have a neural prosthetic device implanted in a region of the brain where intentions are made, giving him the ability to perform a fluid hand-shaking gesture, drink a beverage, and even play "rock, paper, scissors," using a robotic arm.

Neural prosthetic devices implanted in the brain's movement center, the motor cortex, can allow patients with paralysis to control the movement of a robotic limb. However, current neuroprosthetics produce motion that is delayed and jerky--not the smooth and seemingly automatic gestures associated with natural movement. Now, by implanting neuroprosthetics in a part of the brain that controls not the movement directly but rather our intent to move, Caltech researchers have developed a way to produce more natural and fluid motions.

Designed to test the safety and effectiveness of this new approach, the clinical trial was led by principal investigator Richard Andersen, the James G. Boswell Professor of Neuroscience at Caltech, neurosurgeon Charles Y. Liu, professor of neurological surgery, neurology, and biomedical engineering at USC, and neurologist Mindy Aisen, chief medical officer at Rancho Los Amigos.

Andersen and his colleagues wanted to improve the versatility of movement that a neuroprosthetic can offer to patients by recording signals from a different brain region other than the motor cortex, i.e., the posterior parietal cortex (PPC), a high-level cognitive area. In earlier animal studies, the Andersen lab found that it is here, in the PPC, that the initial intent to make a movement is formed. These intentions are then transmitted to the motor cortex, through the spinal cord, and on to the arms and legs where the movement is executed.

"The PPC is earlier in the pathway, so signals there are more related to movement planning--what you actually intend to do--rather than the details of the movement execution," Andersen says. "When you move your arm, you really don't think about which muscles to activate and the details of the movement--such as lift the arm, extend the arm, grasp the cup, close the hand around the cup, and so on. Instead, you think about the goal of the movement, for example, 'I want to pick up that cup of water.' So in this trial, we were successfully able to decode these actual intents, by asking the subject to simply imagine the movement as a whole, rather than breaking it down into a myriad of components. We expected that the signals from the PPC would be easier for patients to use, ultimately making the movement process more fluid."

The device was surgically implanted in Sorto's brain at Keck Hospital of USC in April 2013, and he since has been training with Caltech researchers and staff at Rancho Los Amigos to control a computer cursor and a robotic arm with his mind. The researchers saw just what they were hoping for: intuitive movement of the robotic arm.

Sorto, a single father of two who has been paralyzed for over 10 years, was thrilled with the quick results: "I was surprised at how easy it was [to control the robotic arm]," he says. "I remember just having this out-of-body experience, and I wanted to just run around and high-five everybody."

The Surgery

The surgical team at Keck Medicine of USC performed the unprecedented neuroprosthetic implant in a five-hour surgery on April 17, 2013. Liu and his team implanted a pair of small electrode arrays in two parts of the posterior parietal cortex, one that controls reach and another that controls grasp. Each 4-by-4 millimeter array contains 96 active electrodes that, in turn, each record the activity of single neurons in the PPC. The arrays are connected by a cable to a system of computers that process the signals, to decode the brain's intent and control output devices, such as a computer cursor and a robotic arm.

"These arrays are very small so their placement has to be exceptionally precise, and it took a tremendous amount of planning, working with the Caltech team to make sure we got it right," says Liu, who also is director of the USC Neurorestoration Center and associate chief medical officer at Rancho Los Amigos. "Because it was the first time anyone had implanted this part of the human brain, everything about the surgery was different: the location, the positioning and how you manage the hardware. Keep in mind that what we're able to do--the ability to record the brain's signals and decode them to eventually move the robotic arm--is critically dependent on the functionality of these arrays, which is determined largely at the time of surgery."

The USC Neurorestoration Center's primary mission is to leverage partnerships to create unique opportunities to translate scientific discoveries into effective therapies.

"We are at a point in human research where we are making huge strides in overcoming a lot of neurologic disease," says neurologist Christianne Heck, associate professor of neurology at USC and co-director of the USC Neurorestoration Center. "These very important early clinical trials could provide hope for patients with all sorts of neurologic problems that involve paralysis such as stroke, brain injury, ALS and even multiple sclerosis."

The Rehabilitation

Sixteen days after his implant surgery, Sorto began his training sessions at Rancho Los Amigos National Rehabilitation Center, where a computer was attached directly to the ports extending from his skull, to communicate with his brain. The rehabilitation team of occupational therapists who specialize in helping patients adapt to loss of function in their upper limbs and "redesign" the way patients do tasks with the function they have left, worked with Sorto and the Caltech team daily to help Sorto visualize what it would be like to move his arm again.

"It was a big surprise that the patient was able to control the limb on day one¬¬¬--the very first day he tried," Andersen says. "This attests to how intuitive the control is when using PPC activity."

Although he was able to immediately move the robot arm with his thoughts, after weeks of imagining, Sorto refined his control of the arm. Now, Sorto is able to execute advanced tasks with his mind, such as controlling a computer cursor; drinking a beverage; making a hand-shaking gesture; and performing various tasks with the robotic arm.

Aisen, the chief medical officer at Rancho Los Amigos who led the study's rehabilitation team, says that advancements in prosthetics like these hold promise for the future of patient rehabilitation.

"We at Rancho are dedicated to advancing rehabilitation and to restoration of neurologic function through new technologies, which can be assistive or can promote recovery by capitalizing on the innate plasticity of the human nervous system," says Aisen, also a clinical professor of neurology at the Keck School of Medicine of USC. "This research is relevant to the role of robotics and brain-machine interfaces as assistive devices, but also speaks to the ability of the brain to learn to function in new ways. We have created a unique environment that can seamlessly bring together rehabilitation, medicine, and science as exemplified in this study."

Sorto has signed on to continue working on the project for a third year. He says the study has inspired him to continue his education and pursue a master's degree in social work.

"This study has been very meaningful to me," says Sorto. "As much as the project needed me, I needed the project. It gives me great pleasure to be part of the solution for improving paralyzed patients' lives. I joke around with the guys that I want to be able to drink my own beer--to be able to take a drink at my own pace, when I want to take a sip out of my beer and to not have to ask somebody to give it to me. I really miss that independence. I think that if it were safe enough, I would really enjoy grooming myself--shaving, brushing my own teeth. That would be fantastic."

"The better understanding of the PPC will help the researchers improve the neuroprosthetic devices of the future," Andersen says. "What we have here is a unique window into the workings of a complex high-level brain area, as we work collaboratively with our subjects to perfect their skill in controlling external devices."