IE 11 is not supported. For an optimal experience visit our site on another browser.

Virtual reality helps autistic kids develop skills

A playmate named Sam, a talking dog named Buddy and an Israeli street leading to a Toys"R"Us store all have starring roles in a new generation of virtual reality games designed to teach basic safety and social skills to children with autism.

A playmate named Sam, a talking dog named Buddy and an Israeli street leading to a Toys"R"Us store all have starring roles in a new generation of virtual reality games designed to teach basic safety and social skills to children diagnosed with autism.

For school-aged children with autism spectrum disorders, including Asperger’s syndrome, skills often taken for granted can be torturously difficult, whether staying within the confines of a yard, crossing a street or navigating the social norms of group playtime.

Aided by the observation that autistic children relate especially well to virtual reality and computer programs, an entire field of research has sprung up in the last 15 years. No one has a definitive explanation for the technological attraction, though Justine Cassell, director of Northwestern University’s Center for Technology & Social Behavior, suggested that a virtual reality program’s infinite patience certainly helps.

“Technology is also more predictable and controllable,” she said, “and autistic children like predictability a lot.”

Even so, a particularly convincing virtual reality simulation can help an autistic child transfer new skills back to a real situation — a common difficulty among children with autism, who often focus more on details than on context. New technology can provide an in-between setting, “particularly in a virtual world that’s not moving as quickly and doesn’t have as many cues that might distract you,” said Gary Mesibov, director of the University of North Carolina’s Division TEACCH, a program focusing on the treatment and education of autistic children.

Crossing the street — safely
For researchers at the University of Haifa in Israel, a somewhat pared-down urban street scene has proven its worth in teaching moderately functioning autistic children how to safely cross the street, gaining more independence in the process. Naomi Josman and colleagues created a virtual crossing through a divided roadway, eventually leading to a Toys"R"Us store. The program kept track of how often its six participants, aged 7 to 16, looked both ways before crossing, whether they observed approaching cars, how often they crossed during a red light, and how many virtual accidents they had.

Josman, a professor of occupational therapy, said the kids immediately took to their virtual reality practice sessions, and become increasingly focused. They also improved noticeably. At the onset of their training, the students averaged a score of only 2.66 out of 9. By the end, nearly all had advanced to the final level of difficulty, achieving an average score of 8.91 and reducing their cumulative accident tally from 22 to zero.

But would those improvements translate back to the real world? To find out, the Israeli researchers videotaped the children both before and after their virtual training as they walked within a park that uses traffic lights and small cars to simulate street crossings in a controlled environment. One low-functioning eighth-grader in the study, a 16-year-old boy named Ben, essentially ignored both his teacher and a traffic light during a visit before his virtual reality training.

“At the end, he was standing and waiting for the green light to come,” Josman recalled. “It was really very, very impressive.”

In all, three of the six students were able to transfer their virtual reality skills to the park’s street crossings. The results are in press in the International Journal on Disability and Human Development.

Josman said her team is now upgrading the virtual reality system to make it more user-friendly and customizable, a process she expects to last another six months. The new version will add more skill levels and vary elements like the available light, the speed and number of cars and the crosswalk locations.

Guided by Buddy
Dorothy Strickland, who completed some of the first studies on virtual reality and autism while a computer scientist at North Carolina State University in the mid-'90s, has since developed a range of software programs that feature cartoon characters teaching autistic children how to respond to everything from a fire to a smile. Nearly a dozen options are now available from her Raleigh-based company, Virtual Reality Aids, Inc., and its Web site,

In a program called “My Yard,” an expressive dog named Buddy leads children through a circus-themed game that reinforces the importance of staying in their yard. “Afterwards, they’ll say, ‘Buddy showed me how to do this.’ They make a connection with the character,” Strickland said. Among the participants in her multiple study groups who can use arrow keys on a keyboard and learn in a virtual work space, 80 percent have been able to transfer their new skills to the real world after only 20 or 30 minutes of virtual learning.

For older, high-functioning children, many of whom are mainstreamed in schools, impairments in communication, social interactions, imagination and flexibility can lead to depression and isolation. In response, Strickland’s group has designed programs leading them through the tricky challenge of correctly interpreting and responding to social cues, whether in facial expressions or postures.

Strickland’s team is also revisiting research aimed at helping autistic children master restaurant scenarios, an effort that uses Flash-animated videos to explain what normally happens in a restaurant and why. When they’re ready for a one-on-one interaction, they can enter a virtual space with a behavioral therapist. To facilitate those sessions, Strickland’s group is teaming up with the University of Florida’s Digital Worlds Institute to create safe practice spaces within the virtual world Second Life. A second phase of the project may include up to 20 social settings, including virtual classrooms, homes and movie theaters.

A virtual playmate
Within Cassell’s Northwestern University research group, social interactions and communication skills among children with high-functioning autism and Asperger’s syndrome fall under the watchful eyes of a virtual peer named Sam.

“It’s not a tutor that teaches. It plays the role of a peer that models appropriate behaviors and tries to evoke those from autistic kids,” Cassell said. In a playroom-lab, Sam takes the form of a life-sized cartoon of a typical 7- or 8-year-old child. The image, displayed on a large drop-down or plasma screen, is projected against a background that matches the actual room, creating the illusion of a shared space.

“It has a computational intelligence that allows it to respond to children’s behaviors with behaviors of its own,” Cassell said of the gender and race-ambiguous character. Given the limits of current technology, especially unreliable speech recognition programs for children, much of Sam’s intelligence is controlled by researchers in a “Wizard of Oz” setup, in which they literally hide behind a curtain with a control panel in hand.

A miniature wooden castle anchors the play session, with its front end in the real world and its back end appearing in Sam’s virtual world. Children in the study group can play with three figurines, while Sam’s eyes track their movements. And when a child puts a figurine in the castle’s “magic attic,” the doll reappears in Sam’s hand. “It gives the impression that the real children and virtual child are sharing toys,” Cassell said.

The “mixed reality” environment may help some children overcome the difficulty of getting a lesson in a virtual setting to stick in the real world. It helps, of course, that Sam is an eager playmate, featuring both a voice and a story-telling style typical of children. “Sam can tell a bit of a story and say, ‘Now it’s your turn,’ and the real child is invited to play,” Cassell said.

Although the study is still preliminary, Cassell said observations of six autistic children aged 7 to 11 during half-hour play sessions suggest they were significantly more likely to respond to Sam’s invitation than one from a real child. Importantly, the autistic children’s story narratives also made more sense within a two-way conversation, or were more contingent, in Sam’s company. Plus, they became more and more engaged as they played.

For the project’s next phase, Cassell wants to give the children more opportunity to mold Sam’s behavior, first by operating the hidden controls themselves — and then by designing and naming their own control panels based on which options they think are most important.

The results, Cassell said, could shed new light on why communication is so difficult for autistic children. Do they lack the basic skills, or is something else inhibiting them?

Although her early evidence is only anecdotal, she said, “what we’re finding is that some of the kids that we put in front of these control panels very much have the skills, when the engagement happens via a stand-in virtual child.”

Perhaps as a proxy, Sam removes some of the stress of being with another child. The University of North Carolina’s Mesibov said he found much the same thing when he asked a costumed attendee at a program-sponsored Halloween party why the annual functions always seemed to work the best among young autistic adults.

The articulate young man’s reply: “It’s easier for me to play the part of somebody else than myself.”