Title: Evaluating a Video Enhanced Virtual Reality ... - Do2Learn

quartercircleAI and Robotics

Nov 14, 2013 (3 years and 7 months ago)

106 views


Video Enhanced VR for
Teaching Restaurant

Skills to

Children with Autism


Abstract



While Virtual reality has been used for various training applications, few
studies have measured its effectiveness in teaching social interactions. This
research used

web delivered gaming

technology to create

virtual
worlds where children with Autism Spectrum Disorder

(ASD)

interacted

with avatars
to
practice

restaurant social skills
.

Videos
of similar real world situations

within the virtual sequences

reinforced the lessons and aided in generalization
,
a known
problem for individuals with ASD. Before training, two unknown
restaurant social skills were identified for each of five children with ASD,
aged 7 to 16.
All children who completed VR training correctly
performed two new social skills in a virtual restaurant. In post
-
training real world restaurants, these children exhibited at least
one new appropriate social interaction and 75% demonstrated
2 new social skills.

Learning Steps for Autism


Define a common social skill

deficit
-
eating in a restaurant


Divide social skill into
discreet steps

(1) Wait to be seated, (2) Answer hostess questions, (3) Follow hostess to table, (4) Wait
for server and read menu, (4) Order food, (5) Table conversation


Define measurable

actions in each step

Ex: Step 1 includes determining if there is a hostess or you seat yourself, waiting in the
appropriate place, recognizing the hostess, and waiting for others already in line


Identify
meaning
behind each step
-

Theory of Mind


Show a range of appropriate
responses

for each step


Use non
-
threatening
practice

of steps with avatars in VR


Display
video

of real people in same step within VR lesson


Reward or correct

at each step based on child’s actions


Vary scene/avatar actions to
teach variations


Allow
user control

of steps for different learning abilities


Vary avatar responses to
avoid patterns

and teach concepts

Program Design


Text with video explanation of each step

What to do (Question/How to Respond)


Explain the social action or exchange

Why?
-
Explain the meaning behind the actions

See it!
-
See a video of real people doing the step

Oops… What to do if something goes wrong

Show possible variations of ‘What to do’



Virtual world practice in pizza restaurant

Three windows on screen:


Interactive VR world with avatars


Question/answer and video/correction window



Displays responses child can pick



If child picks correct response, shows video





If child picks wrong response, corrects/repeats step


What to do/Why? Guidance



Multi
-
player practice with other people over web

Program Details


Player moves in virtual worlds with mouse or keyboard directional keys


Avatar program tracks and verbally responds to different actions of the player


Avatars in virtual world include player, a friend the player is eating with, a hostess, a
waitress, and other patrons at different tables


Visual cues. Example: Colors visually separate window functions; Error/reward
statements in red, questions with choices in blue, inactive window in black


Videos at each step are repeatable with button control


Avatar spoken words have both text displayed at window bottom and audio


Print out of step information available for guidance if child prefers hard copy


Text with video explanation program was developed with Flash


VR practice world was created and delivered over web with Wild Tangent Platform


Multi
-
player VR practice was created and delivered over web with Adobe Atmosphere


VR actions are controlled with JAVA programs


Graphics used Wild Tangent world creation toolkit, Atmosphere world creation toolkit,
3d StudioMax, Poser, and Photoshop


Video production was done in a real a restaurant during lunch hour. Waitress, hostess,
and friend were scripted actors. The regular restaurant clientele was part of the filming
background. Fourteen action and seventeen avatar phrase variations were used


Video processing was done with Premier


Study Design


Six children ages 7 to 16 with ASD chosen

All in special education programs and recommended by therapists.

Child 01(Age 11,
Male, Asian)
-
Child 02(Age 7, Male, White)
-
Child 03(Age 9, Female, White)
-
Child
04(Age 16, Male, Black)
-
Child 05(Age 9, Male, White)
-
Child 06(12, Female, Black).



Pre
-
training testing in real restaurants on six skill steps


Individual testing was done in multiple lunch or dinner visits. Restaurants varied and
depended on parent recommendation. Interactions were not scripted. Actions and
latency were recorded by 2 observers. Correct response required that 90% of all parts of
a step be done correctly.

Child had to display inappropriate behavior at more than
chance level for 2 or more steps to be included in study.


Five children practiced correct social actions on computer

Training was approximately one hour, two to three times over a 2 week period. Training
first involved a treasure hunt game to practice navigation in virtual space. All children
mastered control in no more than two plays. Each child then completed the Text, VR
Pizza world, and web training programs. After training, each child assessed his
knowledge with a written questionnaire. All children came back approximately one
week later and repeated the three training programs and questionnaire.


Post
-
training testing in real restaurants on six skills

Pre
-
training testing was repeated in the same restaurants and data collected as before.
Appropriate skill responses varied between visits because hostess, waitress, patrons, and
other details varied, creating different required social interactions. Two restaurant visits
were done per child with an average of 3
-
day separation between visits.


Conclusions


4 children completed study

Child 03 knew all skills in pre
-
tests. Child 06 was removed because of family conflicts


All learned 2 new skills in virtual worlds at 100% accuracy

Four subjects mastered all 6 skill steps to 100% accuracy on all computer training


All did 1 new skill in real world with 90% accuracy

75% did 2 new skills in real world with 90% accuracy

Skill retention was strongest when the restaurant visit was close in time frame to
computer practice. Child 05 did a third computer training between restaurant visits after
missing Step 1 in first post
-
training and subsequently did all skills correctly


Mixed results for one child who generalized only 1 skill

Child 04 was severely autistic (CARS 34) and missed all skills in pre
-
training. In first
post
-
training, he demonstrated 100% accuracy in 5 of 6 steps. In second restaurant visit,
music and the unexpected presence of teacher at next table distracted child. Computer
refresher and retest could not be completed before child moved to another state


Problems

Real world actions did not always match discreet step divisions. Example: child might
order correctly but stare at ceiling rather than waitress (was given 50% accuracy).
Observers required that each action be completely acceptable by normal social standards



Graphs of results and details of pilot study available from
do2learn@do2learn.com