Humanoid robot helps teach children with autism

Aiden, who is three and a half years old, has been diagnosed with autism spectrum disorder (ASD). NAO (pronounced "now") is the diminutive "front man" for an elaborate system of cameras, sensors and computers designed specifically to help children like Aiden learn how to coordinate their attention with other people and objects in their environment. This basic social skill is called joint attention. Typically developing children learn it naturally. Children with autism, however, have difficulty mastering it and that inability can compound into a variety of learning difficulties as they age.

An interdisciplinary team of mechanical engineers and autism experts at Vanderbilt University have developed the system and used it to demonstrate that robotic systems may be powerful tools for enhancing the basic social learning skills of children with ASD. Writing in the March issue of the IEEE Transactions on Neural Systems and Rehabilitation Engineering, the researchers report that children with ASD paid more attention to the robot and followed its instructions almost as well as they did those of a human therapist in standard exercises used to develop joint attention skill.

The finding indicates that robots could play a crucial role in responding to the "public health emergency" that has been created by the rapid growth in the number of children being diagnosed with ASD. Today, one in 88 children (one in 54 boys) are being diagnosed with ASD. That is a 78 percent increase in just four years. The trend has major implications for the nation's healthcare budget because estimates of the lifetime cost of treating ASD patients ranges from four to six times greater than for patients without autism.

"This is the first real world test of whether intelligent adaptive systems can make an impact on autism," said team member Zachary Warren, who directs the Treatment and Research Institute for Autism Spectrum Disorders (TRIAD) at Vanderbilt's Kennedy Center.

The initial impetus for the project came from Vanderbilt Professor of Mechanical Engineering and Computer Engineering Nilanjan Sarkar. His original research involved the development of systems to improve the man-machine interface. He did so by outfitting computer/robot users with biosensors and analyzing variations in various readings like blood pressure and skin response to evaluate their emotional state. The information was used to program computers and robots to respond accordingly.

Six years ago, when visiting his cousin in India, Sarkar learned that his cousin's son had been diagnosed with ASD. "After I learned something about autism, it occurred to me that my research could be valuable for treating ASD," he said.

At the time, several experiments had been conducted that suggested young children in general, and young children with ASD in particular, found robots especially appealing. "We knew that this gave us an advantage, but we had to figure out how to leverage it to improve the children's social skills," Sarkar said.

"You can't just drop a robot down in front of a child and expect it to work," added Warren. "You must develop a sophisticated adaptive structure around the robot before it will work."

To develop this structure, which they named ARIA (Adaptive Robot-Mediated Intervention Architecture), Sarkar and Warren assembled a team that consists of Esubalew Bekele, a graduate student in electrical engineering and computer engineering, Uttama Lahiri, a graduate student in mechanical engineering who is currently an assistant professor of electrical engineering at the Indian Institute of Technology in Gandhinagar, Amy Swanson, a project manager at TRIAD, and Julie Crittendon, assistant professor of pediatrics at the Vanderbilt University Medical Center.

The team decided that a robotic system had the greatest potential working with young children. "Research has shown that early intervention, individualized to the learner's needs, is currently the most effective approach for helping children with autism develop the foundational social communication skills they need to become productive adults," Crittendon said.

So the researchers built an "intelligent environment" around NAO, a commercial humanoid robot made in France, whose control architecture was augmented for the purpose. The small robot stands on a table at the front of the room. Flat panel displays are attached to the side walls. The chair where the child sits faces the front of the room and is high enough to put the robot at eye level. The room is equipped with a number of inexpensive web cameras that are aimed at the chair. Their purpose is to track the child's head movements, so the system can determine where he or she is looking. To aid in this effort, children in the study wore a baseball cap decorated with a strip of LED lights that allowed the computer to infer where they are looking.

NAO has been programmed with a series of verbal prompts, such as "look over here" and "let's do some more," and gestures such as looking and pointing at one of the displays, that imitate the prompts and gestures that human therapists use in joint attention training. The protocol begins with a verbal prompt that asks the child to look at an image or video displayed on one of the screens. If the child doesn't respond, then the therapist provides increasing support by combining a verbal prompt with physical gestures such as turning her head or pointing. When the child looks at the target then the therapist responds with praise, such as telling the child, "good job."

The setup allowed the researchers to test the relative effectiveness of the robot-based system and human therapists in joint attention training with a dozen 2- to 5-year-old children, six with ASD and a control group of six typically developing children. They alternated short human-led and robot-led training sessions and compared how the children performed.

The test found that the children in both groups spent more time looking at the robot than they spent looking at the human therapist. During the human-led sessions, the children in the control group spent significantly more time watching the therapist than the children with ASD did. In the robot-led sessions, however, both groups spent about the same amount of time looking at the robot.

"The children's engagement with the robot was excellent," Crittendon said, "and we saw improvements across the board in both groups."

One of the key elements of ARIA is its closed loop design. The robot adapts its behavior to each child automatically depending on how he or she is responding. "There is a saying in the field, 'If you've seen one child with ASD, you've seen one child with ASD.' So one size does not fit all. To be useful, the system must be adaptive," Warren said.

The cost of robotic systems like this will continue to come down in the future so it should easily pay for itself by supplementing human intervention."In addition, ARIA is not designed to replace human therapists, who are in short supply, but to leverage their efforts. "A therapist does many things that robots can't do," said Sarkar. "But a robot-centered system could provide much of the repeated practice that is essential to learning. The cost of robotic systems like this will continue to come down in the future so it should easily pay for itself by supplementing human intervention."

Warren hopes that robotic systems can act as an "accelerant technology" that actually increases the rate at which children with ASD learn the social skills that they need. Encouraged by the success of this current study, Sarkar and Warren have started developing robot-mediated autism intervention systems that will address other deficits of children with autism such as imitation learning, role playing and sharing.

The research was supported by a Vanderbilt University Innovation and Discovery in Engineering and Science (IDEAS) grant, National Science Foundation award 0967170, National Institutes of Health award 1R01MH091102-01A1 and by the Meredith Anne Thomas Foundation.