Chatter bots in a Virtual World

estonianmelonAI and Robotics

Oct 24, 2013 (3 years and 9 months ago)


University of Arkansas

CSCE Department

CSCE 4613 Artificial Intelligence

Final Report

Fall 2009

bots in a Virtual World

Jordan Gilbertson and Matt Burton


Currently the hospital simulation we have in Second Life has very little acti
vity and a very little
communication among its resident
avatar bots

and smart objects. Our project aims to overcome
this defect by introducing chatting entities into the hospital. Our goal is to create a script which
can be attached to any object in Seco
nd Life and which will cause the object to immediately
become a chatting object. So far we have found that we need to work with existing chat
code and modify it to be useful in the hospital setting and then find a way to link the new chat
bot into Sec
ond Life.



1.1 The


In the future, where “everything is alive” (EiA), all objects will have identity, an ability to
communicate, and a way to interact with other objects and humans. Without the ability to
interact through spoken, or

written, language the objects in the hospital will not be able to
effectively communicate in an EiA setting. They can, of course, communicate through other
means via the 'magic' of computing but this will not play well to the EiA idea. In a real hospita
or any real human setting there are people who talk. Our main form of communication is the
spoken language and as a result we want to see entities in the hospital communicating.

This provides many challenges as modern artificial intelligence has not yet

'solved' the spoken or
written language understanding problem. Repeating a script is simple, but getting objects to
interact in a lifelike matt
er is not. Chat bots need
excellent natural language processing to
understand what is being asked and to respo
nd intelligently. This two
sided coin presents many
problems for programmers and it has been worked on since the early 60's with only moderate

1.2 The


The objective of our project is to create a script in Second Life which can then be a
ttached to any
object thereby turning that object into a chat bot. We want to link the script to a chat bot
program hosted elsewhere where it can be easily modified and developed for future use.

1.3 Context

The objective of the

Everything is Alive

ct at the University of Arkansas is to create a
world where everything is both interconnected and interactive. The inter
connectivity will have
advantages such as being able to do just about anything from just about
. The
interactivity will have
advantages of being able to get just about any information from just about
anywhere. These of course are only two examples of the advantages of a world where every
object can communicate both with humans and with other objects.

At the moment pervasive com
puting is an idea not fully realized. With 3D virtual worlds it is
easier to get an idea of what pervasive computing may one day mean. With simulations such as
the hospital we can begin to see what exactly will happen when

Everything is Alive.

What is

the full potential of such a scenario? What disadvantages are there? Using virtual worlds we are
able to get an idea about these things before they are here. With chat bots in the virtual world
we are hoping to get an idea of how things will interact
once both objects and humans in the
virtual world can talk to one another. As spoken language is the primary form of communication
human to human it is very important to see how, in a pervasive computing environment, the
ability for objects and humans to
talk to each other would play out.

1.4 Potential


Chat bots hold endless possibilities to solving everyday lives both in the virtual world and in the
real world. A chat bot in the Second Life world of the University of Arkansas might work for
the vi
rtual hospital and help give advice to patients without the use of an actual real life doctor.
The same could be said for the real world. Chat bots might also help keep people company by
giving someone something to talk to and carry out a conversation wi
th. Currently chat bots hold
a major impact already with customer service departments within companies as they help limit
the flow of customer problems to what might be a short staffed customer service line keeping
both the employees happy with fewer call
s and the callers happy with shorter wait times.
Another potential impact of our project is aimed to helping build a chat bot to help speech
pathologists in training learn how to speak to people with a speech disability. In order to do this,
however, spe
cial speech recognition software would need to be implemented to read in the
trainee’s speech to text as well as translate the computer’s test output to speech with disability.
With a growing computing community, it would be a shame not to take advantage
of Artificial
Intelligence in the chatter bot realm to help our daily dilemmas, especially those already

2. Related Work

2.1 Key


The main related technologies are of course previously developed chat bots such as ELIZA [3],

], A.L.I.C.E. [5] and Jabberwacky [6]. The short comings of these technologies are
all the same. They simply cannot yet pass a Turing test unless the user queries are very narrowly
defined. For the most part these technologies are based on pattern match
ing and have very little
or no reasoning involved. Jabberwacky is the sole exception to this as it can 'learn' as you speak
with it. It does this by storing all user interactions with it and attempting to find more
appropriate responses.

Other technologi
es include natural language processing which is a main off shoot of artificial
intelligence. In order to get chat bots working properly they will need the ability to in some way
understand what humans are saying. Virtual worlds provide a place where we c
an test out the
chat bots and see how they interact. Pervasive computing will find chat bots useful in that they
will provide an incredible way for objects which are alive to communicate both with each other
and humans.

Related Work

Many chat bots hav
e already been created

and developed already
Most chat bots these days use
a chatterbot brain that relates back to Eliza, Parry, Alice or Jabberwacky as they are some of the
most advanced bots programmed to this date and provide for a structural basis fo
r teaching new
bots how to talk. Also, s
2.1 Key Technologies

for Eliza




[6] in references.

Robust Sentence Analysis and Habitability, A thesis concerning the topics of natural language
processing and the habita
bility problem. [7]

The iPhone now uses voice recognition software. [8]

“Mybotai” is an AOL Instant Messenger screen name that is actually a bot. When talking to
mybotai it almost seems as if you are talking to a real person. It is the best bot I have
untered to this day and is the learning summation of many years of work. When talking to
him I couldn’t even find out what kind of chatterbot brain he had because he stated he was better
than the rest.

Related EiA Projects

Our projects on Chat bots rel
ates to these other projects, e.g.,

Mirror Worlds project

our project develops a script which can be used to make objects
in the mirror world talk. This helps emulate the real world.

However, with the time
given in a semester and our limited knowledge

of Action script and Linden Script we
were unable to port our project into Second Life, leaving it for future work and

Ontology project

our project develops a script which covers the hospital chat ontology.

Soft Controller project

using c
hat bots soft controllers could one day talk to their users
making information exchange quicker and easier.

Smart Devices

similar to soft controllers our chat bout could one day be used to enable
smart devices to talk to humans and to other smart devices

Along with controlling these
devices by being asked to turn something on or off a smart device might also brag about
its capabilities and let you know what it might be able to do.


again a chat bot with good natural language processing could b
e used to
quickly and effectively parse spoken commands and to communicate what needs to be

Search Spider

with a proper natural language processor a search spider could listen to
conversations and parse important information.

Games in SL

the abil
ity of the game to talk to a person via our chat bot could be used to
enable blind people to play games in Second Life and as with other areas it would make
information exchange easier and quicker.

3. Architecture

3.1 Requirements or Use Cases

(Key: gre
en = accomplished, red = future work)

Acquired Background


earned how to Program in Flash Action



Learn how to Program in llscript

(Action Script was challenging enough)



how to write in XML scripts

Chatbot code
written and educated (th
rough XML sheet and actual program


Project FranX is usable although it needs to be ported/socketed for use with
Second Life input/output


Project FranX is able to be educated through XML sheets

Project FranX needs to be more capable of muli
dimensional arr
ays. See
below in Architecture for reasoning why.



Healthcare information into Project FranX

Chat bot was re
written with a symptomChecker() method that would
check your symptoms you inputted and compare it to problems bigger
than your symptoms (
common cold, sinus infection, flu)

Needs more symptom resolution but now has the ground work

Needs more advanced education


Program Project FranX to stutter

Student, Abbi Wood (
), would greatly appreciat
e any
kind of development she could have towards having a stuttering chat bot
for her study

Chat bot is ran on own personal machine through Flash Player


Program must be launched in same folder as the XML education sheet


Need to convert the code out of Acti
on script and into another language more
easily worked with as well as host on a server with a communication port opened
for Second Life

code Script for objects in Second Life


We can place this code on bots or objects


This will enable anything to talk

and interact bringing it alive


Have script port out to Project FranX


Make the bots appear to be real persons communicating with you

Try to implement
smart capability to pass Turing test


Enable bots to talk to each other

Bots must be able to initiate their

own conversation when they sense
another bot is around

Design Space

The architecture of our project can be split up into a few main components that drive it. The first
core component is the chat bot shell
. It is made up of Flash Action
Script that operates the

functionality of the bot and operates its speech patterns based off of a XML sheet
that educates the bot into what to say.

In order to access this Action Script you must open the bot “.fla” file in Adobe Flash (we used

CS4 in the Union and J.B. Hunt GACL on the Macintosh machines). Once you open the “.fla”
file you need to make sure that the first layer is selected (the actions layer) and then go to
“window” in the top menu and pull down the actions pane. Now you’ll s
ee an extensive list of
code that runs and operates the bot.

The main education of the bot is hosted within the XML sheet. In order to educate the bot, you
must make a parse tree for the bot to search through. For example, <parse1><parse2> and when
you a
re ready to define the education within the parse words you must place before the education
<answX> so that the code knows that this is an answer. The problem we realized with the
parsing is that the code in the action script only allows for a bi
nal array meaning that
the bot can only parse two words. A tri
dimensional array might prove to be more effective with
the bot education as some questions are a bit vague with two key words instead of three. For
examples of this you may look over our XML

sheet and see which responses we wanted to parse
more words into to make it a more effective communicator.

If a chat bot wanted to learn what it was talking about and reference it in the future our chat bot
would need a new XML sheet and implement a new f
unction inside of the Action Script that
allows for the bot to input into the XML sheet any topics it might not know about so that when
asked about the topic in the future it might have a clear understanding of what it is then saying.
Another useful idea
for educating a bot through the bot’s own self
education just mentioned
would be having a trainer for the bot. This would be someone who could talk to the bot just to
educate it and trigger these learning functions inside of the bot. This would have to b
e someone
that is trusted so that the bot does not grow corrupt in its own language. A learning bot is like a
child. Its formal knowledge of communication is very primitive and growing. A learning chat
bot will pick up new phrases

the same way it was ta
ught. If you teach a kid a word that is nasty
or dirty then the kid might repeat this later. You are effectively doing the same with the bot.

The other core component of the c
hat bot would have been

the Linden Script that enables any
object or bot with t
he script running to listen for people chatting with it as well as drive
responses from a server. The main concept in this project is to connect the script with the chat
bot shell. The script can listen for active conversation but it cannot read and resp
ond. The chat
bot shell can communicate with people but not without a main driving interface that listens for
active conversation. In order to design these
bots the Linden Script and Action

Script will have
to be ported so that the shell accepts incoming

language from the game as well as decipher and
rebound the response in an outgoing port back to the game. The linden script will listen for the
text and pass it along to the server rather than try to read and respond by itself.

With hosting the actual ch
at bot on a server it will make it possible to drop the linden script on
any object that we desire to have it talk about healthcare to us.

The difference between our chat
bot and a chat bot that only knows general things (such as a smart controller) is th
at our chat bot
is fairly extensively educated with over fifty medical responses that have been added to its XML
sheet as well as being able to make reasonable assumptions about your symptoms to draw up
what might be wrong with you.

The last thing about ou
r chat bots is that we eventually wanted it to be able to or have the option
to stutter so that it might assist others with learning how to communicate with people with a
speech disability. Such problems with stuttering include repeated first consonant, r
epeated word,
block, as well as any starting and stopping in the middle of a sentence. The chat bot would also
need to have a text to speech interpreter that was programmed with these speech disabilities to
make communication and education with a stutter
bot more effective. The stuttering would also
need to be fairly random and only happen so often and only on certain syllables.

. Tasks

The first task is

to review related work. This will develop research for the creation of a
healthcare chat bot. This

esearch has been completed by both teammates more specifically the
history behind chat bots and how they have evolved by Mr. Burton and chat bot script and code
by Mr. Gilbertson.

Once sufficient research has been completed it is determined that both t
eammates must learn the
language or already know the language that their project is or is to be coded in. In ou
r case we
both must learn Action

Script and Linden Script.

At the end of the semester it was later
determined that only Mr. Gilbertson needed t
o understand Action Script so that he can tweak the
bot to the ways and functions that we need. Also XML sheets and how the code used them was
needing to be understood by both team members so that they can continue with educating the

When the program
ming language is understood the teammates

the code found

by Mr.
Gilbertson and
split up the fixing and replacement of code between both members by assigning
one member to fixing up the bots main code

that still needs

to be ported to Second
and th
e other
member of the team assigned to educating the bot on the XML sheets with the assisted
understanding of the code by the first team member.

Once successfully ported, both teammates will need to educate the bot. The education of the bot
cannot be done

by one person alone as it not only requires a multitude of people programming
infinite data into bots but as it also requires speaking with healthcare professionals to find out
what the scope of healthcare is that we need to incorporate in the education o
f the bot.

To further
explain this after the project the bot was educated with very basic health terms. However, the
bot was never able to get to port to Second Life, even after asking Mr. Keith Perkin’s assistance.
His assistance was still helpful none

the less as he laid the need to have this program re
to Java or some sort of C language so that it can be more easily manipulated.

When the bot is educated and tested more research will be needed to learn how to make the chat
bot stutter and help
other research students in need.


The testing with our work is simple. Chat bots talk to people and things and are “people” and
things. So to test our solution we must talk with our chatbot.
To talk with our chat bot we must
open up the flash

video file that runs the bot, sign in with our name and start chatting away. To
extensively test the bot we ran through our XML sheet and went down the parse tree to test for
correct responses. If the bot did not produce correct responses we later went
back into the
Action Script and would determine why the code would not go through all of the parses. Aside
from multi
dimensional arrays past two dimensions for the code it was also determined that for
proper parsing the first word of input MUST be the fi
rst parse word. Past that you can have
whatever other words you would like between your parses.

4. Results and Analysis

We had

been playing with some chat bot linden script that communicates with a public server
outside of our own control and modificatio
n. We have been playing with this script to better
understand the project we have to do and how we need to modify the code for our own doing.
Currently we have the original script operating on a couch in my floating house. It talks back but
the language

is somewhat crude and needs further development as well as specialization in
healthcare. The script itself is A.L.I.C.E. based and hosted at



Our other code development is finding a chat bot shell or program that is free and open sourced
so that we can educate and modify the program as needed. We found Project FranX as a free
open sourced code that has the permission
of use granted that the use is not widespread and that
it is not used for profit.
We have re
worked the code from its original state into something newer
with updated graphics as well as education and symptom checking.
The below screen shot


the left

s a demonstration of the

chat bot interface.
We have been able to educate the
chat bot but w


also hoped to change it by allowing the bot to interface

with Second Life
instead of HTML
The below screen shot on the right is a more curre
nt screen shot of our
chat bot after working with it to update the visual appearance as well as
teaching it to talk.
can be downloaded from


5. Conclusions


In conclusion we understand where our project must go to from here. We have successfully
researched enough to find sufficient code to have an operational bot as well as a sufficient
interface to Second Life. We
can communicate currently to Project FranX through HTML
and speak to a couch in Second Life through script that connects to an outside website hosting
chat bots.

Our chat bot is also educated fairly well with basic healthcare knowledge to tell you
hat you need medication or sleep (as well as a few other things).

Future Work

We still need to connect the Second Life interface with the chat bot.
did not permit us to
allow the chat bot to stutter in its speech. Someone else

then we will also b
e needing to make the
chat bot stutter at a fairly realistic rate to help other researching students understand people who
stutter and why.


Jordan Gilbertson

Mr. Gilbertson is a senior Computer Engineering major in the
Computer Science and Computer
Engineering Department at the University of Arkansas.
He has completed Digital Design 1/2, FPGA/CPLD’s, Programming Foundations 1/2, as
well as Calculus 1/2/3. He is currently a Client Services Intern for J.B. Hunt
Transportation Inc. Mr. Gilbertson
is r
esponsible for the following:

Understanding the chat bot code in Action script and XML for the team’s gain

This helps Mr. Burton to educate the XML sheets

This allowed for Mr. Gilbertson’s development in a symptom checker

Cleaned up the code as the white s
pacing and format was inconsistent and unreadable
at first. Now the program is formatted correctly with visual neatness and ease of

Also updated the XML sheet with some basic health care questions and responses

Developed a Symptom Checker that go
es through your symptoms listed and tells you
what kind of problem that you have.

Matthew Burton

Mr. Burton is a senior Computer Science major in the Computer
Science and Computer Engineering Department at the University of Arkansas. Mr.
Burton is respo
nsible for the following:

Updating the FranX bot with basic health questions parsing for question and

Research concerning chat bots, habitability, and natural language processing.

Dr. Craig Thompson, Mentor

Thompson is a professor in the Compu
ter Science and
Computer Engineering Department. He leads the Everything is Alive research project
that is currently focusing on how to simulate pervasive computing using 3D virtual
worlds. See



Project Chatterbot:
FranX Bot,


Pandora Bot for Second Life,


Eliza chatbot,


PARRY chatbot,


ALICE chatbot,


Jabberwacky chatbot,


Habitability and Natural language processing,


iPhone and Voice Recognitio

Appendix A

Deliverables Manifest

ed in Chat


The XML sheet responsible for bot education


The actual flash file for editing the bot


The flash move file compiled from code for execution



html compiled from code for execution


Our Final Report