Real Change from a Virtual Classroom

Today is just another school day for Professor Fengfeng Ke’s students. After boarding a bright yellow bus, they drive down the familiar streets until they reach the school. There’s a particular excitement today, as they have a field trip to one of their favorite destinations: a western town, complete with Conestoga wagon, general stores, and dusty streets. But first, the students have to get through the school day.

As the students exit the bus, a teacher appears. Literally, she appears out of thin air.

None of the students are fazed by this aberration. These students, who have volunteered for Ke’s research, see people appear and disappear all the time; that’s because, unlike typical students, these students are exploring a virtual world, where buildings and people pop-in as they load into the program. Each student sits behind a computer, controlling a virtual avatar, communicating over voice chat or text chat, and emoting using a list of pre-defined gestures.

Another difference between Ke’s student volunteers and a traditional classroom? All of these students are also diagnosed with high-functioning autism, formerly called Asperger’s Syndrome.

Once or twice a week, these volunteer students meet with Ke and her team. The students are generally in 5th-7th grade. Ke, who is an associate professor in the Department of Educational Psychology and Learning Systems, carefully observes the students via her own in-game avatar. As both observer and creator of this virtual world, Ke hopes that she can build much more than a virtual playground; she hopes that she has created a safe environment for children with high-functioning autism to practice their social skills.

The idea to create a virtual training environment came to Ke after her experience teaching a graduate student with Asperger’s. The student was excellent at coursework but suffered in navigating social situations both in and outside of the classroom. “Because of his unique way of communicating, he couldn’t market himself like other people,” Ke says. “It made me feel a little bit frustrated because I knew he was wonderful and capable of coming up with wonderful instructional design projects.” The student would receive interviews but had difficulty advancing further in the job process because of his difficulty in communicating.

While no two cases of high functioning autism are entirely the same, struggling to communicate is a common problem for those diagnosed. Children with Asperger’s struggle to recognize the subtle shifts in tone and speech that convey emotional meaning. As they get older, they might understand social interactions better but may still have difficulty interpreting more complicated communication, such as nonverbal cues. Like many disorders associated with the autism spectrum, there is no cure for Asperger’s, although early intervention and support programs can help improve outcomes.

Ke envisions virtual reality as one of these resources that could potentially help children with high functioning autism feel more comfortable communicating. The program is easy to use, secure, and safe, since Ke and her team control who can access the virtual world. Each child receives a secure login to the program, which can be opened on nearly any computer with an internet connection. After logging in, children control their avatar, which they have customized, to participate in the day’s activities. These activities allow children to practice social interactions with each other, instructors, and non-playable characters (NPCs), which can be scripted to perform basic routines or puppeteered by one of the instructors. Whenever possible, Ke and her team try to schedule sessions so that children interact with one another.

For the purpose of Ke's research, the children engage in focused gaming and roleplay, as well as collaborative design. Focused gaming and roleplaying includes recreating common social interactions within the program. For instance, students might interact with others in a virtual cafeteria or library. They can also participate in games within the virtual world, like the traditional Chinese Game, Go. While they engage in this roleplay or play games, students are practicing their communication skills. These exercises are particularly useful, since one of the most common symptoms of high functioning autism is a tendency to focus on one's self and engage in one-sided conversations. Navigating a game's rules or roleplaying forces the students to think about the other participants in these social exchanges.

Students also engage in collaborative design, which especially leverages the unique possibilities of virtual reality. In these instances, students can change the virtual world around them, designing buildings and making important decisions. Because the platform is open ended, students and instructors work together to build the world, and have created a bustling virtual landscape across a number of settings, including a western world, an underwater city, a ski resort and more. The catch with collaborative design is that students must work together, which can pose a challenge for children with high-functioning autism. For instance, if a student wishes to make a new bus stop, they must talk to someone acting as a designer to agree upon a design, and then they contact the contractor to place the structure in the world. This forces the students to interact with others and, again, practice their social skills, learn how to negotiate, and manage the opinions of others in their groups.

Ke has found that students love designing the world around them— almost as much as parents of these students love the results. Ke has analyzed performance and behavioral data, as well as self-reported data and parental-reported surveys. “Based on pre- and post-comparisons and also cross-subject comparison, we are confident that VR helps,” Ke says. “We also interview the parents informally, either face to face or conferencing on video chat, to understand how they feel their kids are changing, and so far, most of the feedback is positive. [Their children] have been much more competent in being able to initiate social interaction.”

“A common pattern is that children with high-functioning autism are not motivated or interested in initiating conversation,” Ke continues. “Sometimes they’re not skilled in initiating in conversation naturally with strangers or people they don’t feel comfortable with. This is the part where we feel like VR has really helped a lot.” Their excitement for this virtual world translates into an overall improvement in their social skills in the real world.

A possible reason for this increased willingness to initiate conversation, Ke suggests, is the additional avenues of communication inherent in virtual reality programs. While popular social media platforms like Facebook focus on written communication, for individuals with high-functioning autism, text-based communication can limit their ability to connect. If those individuals are comfortable communicating through text, then this limitation is not a problem, but for those who prefer a different means of interaction, they can either struggle through text-based communication or simply not engage. Virtual reality, comparatively, has a number of ways to communicate since it is a much more visual way of communicating. Through emotes, participants can perform non-verbal gestures that are easy to understand and are standardized, limiting some of the nuance that makes real-world equivalents more difficult to decipher. They also can communicate via voice or text chat. This multi-method communication channel opens up the opportunity for socialization.

Virtual reality also offers an avenue of self-expression that other communication channels lack. For instance, Ke’s research group have responded strongly to customizing their own avatars. Children choose their outfits and overall appearance to their liking. This positive self-identity encourages children to interact with others and gives them confidence.

Interestingly enough, it is also the limitation of technology that has helped some of Ke’s students become acclimated to methods of communication that they might not otherwise feel comfortable using. “At the beginning, sometimes the children are nervous about typing and they don’t want to type,” Ke says. “However, if one of the kids has a headset that does not work or the volume is too low, they have to use the text chat. We notice that they are willing to do so.” Regardless if everything works perfectly or if students have to adapt to the technology, Ke sees virtual reality as an effective tool to help those with high-functioning autism understand different communication strategies. “Sometimes [the students] tend to be dominant. ‘I have a good idea and I think it is good,’” Ke says. Their interactions devolve into “my way or the highway” moments. Her virtual reality program has helped resolve this ultimatum mentality, particularly after the children engage in a collaborative design session.

They all share some common symptoms and needs, but at the same time, they are so different in who they are and what they need.

Ke hopes her research into virtual reality socialization is just beginning. “Currently we focus on children with high-functioning autism. We think it can be expanded to children on the other end of the autism continuum.” Her current research already hints that virtual reality social training could help children across the autism spectrum. There is also evidence that her virtual reality socialization could help older children; some of the study’s participants are older than their grade level due to being held back in school.

Ke also wants to expand the program to children developing typically. She believes that this would help children understand autism better. “They will understand there are different ways of thinking, different ways of communicating.” Additional avenues of research include integrating motion capture technologies, and Ke and her team have already begun experimenting with using Microsoft’s Kinect–an advanced camera that captures movement and translates it to on-screen animation nearly instantly–in the virtual reality.

As Ke continues to work with the virtual reality program, she hopes to keep improving it. “Right now, when we think about special education research, we still kind of group children and we think they all share the similar patterns or symptoms, which was not necessarily true when I collected the data and analyzed the participants’ performance. They all share some common symptoms and needs, but at the same time, they are so different in who they are and what they need.” Ke and her team continue to refine the program by data mining, allowing them to create a virtual world that better adapts to the needs of those in it. Eventually, Ke hopes she can create a cost-efficient tool that professionals can use to help children across the autism spectrum.

Regardless of where the future takes her virtual world, Ke’s volunteer students will continue to enjoy building a world of their own—and building something even more important: confidence in communicating with those in the real world.