Artificial Intelligence in Gaming

odecrackAI and Robotics

Oct 29, 2013 (3 years and 9 months ago)


Artificial Intelligence in Gaming

Ryan Donnelly

Department of
Computer Science

University of Wisconsin Platteville


Artificial intelligence


in gaming is getting more and mo
re realistic as we head well
into the twentieth century. But will

be at a state where games
replicate humans
perfectly on a consistent basi
s and learn their instincts?

Games are getting more complicated. This is due to the use of more advanced a
that are being used

as well as the new hardware that allows more computations
wouldn’t be where they are today if it wasn’t for artificial intelligence. I will be
discussing the importance of artificial intelligence in games such as RTS
type games,
First Person Shooter
pe (FPS
type) and sports games. I will also discuss the most
popular algorithms used today for artificial intelligence in games such as

A* algorithm,
finite state machines
, and artificial neural networks


AI in gaming refers to methods used to produce intelligence of non
player characters or
computer players. The methods are usually created from existing methods in AI.

may be
methods used but it is a

different approach to common AI.

In first
person shooter (FPS) games the computer is basically cheating due to the fact that it has
to know where you are at all times to try to achieve the most intelligent non
racter. In a lot of cases, the

AI of the non
player character has t
o be lowere
to give the human player

a sign of fairness and a chance to win.

Programmers will want to stay away from cheating as much as possible. In a way this
makes the game have an unrealistic challenge because it is not using intelligen
Cheating also draws away focus from programmers to program more human
like bots.

AI can
be thought as
simulate thinking, or as representing intelligent b
ehavior in
an algorithmic way

When creating AI for games,
one thing should always b
e in the
programming team’s head, “what would a human player do?” The obvious goal is to
make games to have a realistic challenge for the human
. This makes sure that the
game does not get too boring
and shoved under the shelf in a week

Many t
echniques are used to achieve human like behavior for computer players. The
most popular techniques are the A* algorithm, finite machines, and artificial neural
networks. Each contributes to the AI in different ways. Without these techniques AI

be where it is today not only in games but in all other software trying to
replicate human intelligence.


Videogames first entered the world in the 1960s and into the early 1970s with the likes of
, and

. These games
d no

in them but used some
discrete logic because they were based on a competition between two human players.

Artificial Intelligence began to get implemented into games in the 1970s with a single
player mode to play against the computer enemies


The enemy moved by the use of
stored patterns.
Stored patterns are paths that never change

and the computer
follow the same paths
that are stored in the game.
When microprocessors came into play,
building of the AI became more complex due t
o the speed and amount of computation
that the microprocessors allowed.

When the game
Space Invaders

was released in 1978
, it brought to the table distinct
moving patterns,
which is different from stored patterns in that there are different paths
for the
player [1].
The following year,

was released
and the AI in
this game stepped it up a notch from
Space Invaders

in that it allowed for
more comple
and varied enemy movements



debuted in 1980 and was the first popular
game ch
aracter. Pac
Man sold all kinds of memorabilia.

As the years went on,
different personalities for the enemy were

introduced as well as
fighting games such as
Karate Champ

in 1984

A big accomplishment for AI
occurred in 1983

human player went

down in defeat to a computer player in
chess. This was the first victory recorded for AI against a human player at anything
This opened up many eyes in the software industry.

even bigger accomplishment
occurred in 1997 when Deep Blue, a ches
s playing computer developed by IBM,
defeated Kasparov, a world champion chess player


This event gave confidence to
every AI developer to believe that anything can be done

or programmed

to what they
want it to be or do

Sports games

advanced game AI

in the

s with
the use of expert
systems [

An expert system is a program that contains knowledge on a subject of one or more
human experts.
An example would be the popular Madden Football series. The AI of
this game
tried to mock the coachi
ng or managerial style of John Madden himself.
Extensive work
and time put into this game

to try to maximize the accuracy of the AI. In
the newer Madden games one can change the variables of the opponent to play how the
user wants them to play.

Most gam
es now have a difficulty level as well which all
the user to pick a difficulty level which matches their experience with the game.

Finite state machines started to enter the gaming world in the 1990s

as well as artificial
neural networks
time st
rategy and first
person shooter type games first entered the
market in the 90s as well.
The first Real
Time Strategy games had notorious problems
which included

path finding problems, real
time decisions and economic playing

y more. As for an exam
ple, in the game
Dune II
, the enemy
uses cheats and when
the computer
players attack a human base, they attack in a bee line instead of attacking in
random positions


The RTS games did get better in the AI department and have
gotten more popular


instance, one of the most popular games are the WarCraft
series, and in the 1990s the first WarCraft game was the first

to ever implement
finding algorithms at such a big scale for hundreds of units to engage in huge battles


Graphic cards
also come out in the 90s and allow for more processor time to be
spent on AI

What we are seeing now in the

century is that
games are
implementing artificial neural networks
Battlecruiser 3000AD
was the first game to use
neural ne
tworks 1996

That spawned the games
Black & White

controlled characters’ learning was used for the first time

processors came out in the early 2000s as well as just recently the core
duo processors
allow for even more of a complex AI engine to be built.

AI in
First Person Shooter (


In first person shooter
type games the AI is in the opponents, teammates, and extra
First person shooter
type games
can be

created using a l
ayered artificial
intelligence structure. The bottom layers take care of the trivial tasks such as determining
he best path to the target, and t
he top layers deal with reasoning and the
behavior of the
computer player

When a behavior is decided, th
e lower
level module selects the best
tactics for completing a task. After the tactic is picked, the AI of the computer player
determines the best approach for the situation. For example, if the tactic picked is to
fight, the approach could be to sneak u
p on the opponent, hide and wait for the opponent,
or run at the opponent and firing.

An event driven engine
can also be

used where
action is taken based o
n specific events
that happen
. For example, some events with actions could be


If enemy seen
→ Then a
ttack with best weapon

If w
ounded →
Then r
un away or hide and find health

If o
ut of ammo →
Then f
ind ammo

If you take a look at the first Doom game, it would have only one event which would be
“if enemy seen,

The same could probably

be said for the new Doom games
as well.

When using

an event driven engine, it

good idea to use
leaking buckets. This allows
for possibly a different action than a human player may expect.
The leaking buckets
theory is where there are buckets that le
ak some of their contents over time. The script
that gets ran is the script with the most filled bucket

. For example,

s say there
exists the buckets flee, fight, and restock. Events that occur fill the buckets to a different
extent such as

n enemy → Add 5% to flee and 10% to fight

Low ammo → Add 20% to restock


→ Add 20% to flee and 10% to restock

A lot of ammo and

→ Add 50% to fight, remove 20% from flee and restock

Lost 50%

in one hit → Add 50% to flee, add 20% to r
estock and remove 50%
from fight.

After an event occurs, the bucket with the most content in it gets ran. So if the computer
player sees an enemy and has 45% in its’ flee bucket and 30% in its’ fight bucket then the
player will flee instead
of fight.

Most all first
person shooter games use some sort of path
finding system.

The path
finding is based on graphs describing the world where each vertex on the graph
represents a location [2].

The location could be anything related to the game suc
h as a
room or a tower.
When the computer player is ordered to get to a given point, navigation
is done by using the points


it should consecutively head towards to reach the
specified location


The A* algorithm is the most popular algorithm or t
echnique used
for path
finding. It guarantees to find the shortest path between two points.

The animation system or the computer player movement
in first
person shooters
has to
play an appropriate sequence of animation at a chosen speed.

Basically it mu
st be in sync
with the AI system in the game.
Different body parts should be able to play at different
animation sequences such as

for example

a soldier running and aiming at the enemy then

and reload

while still running

. Such games often

use the inverted


which is
the process of computing the pose of a human body
from a set of constraints [4]

In an IK animation

calculations can be

arm positioning animation so that the hand can grab an ob
ject off a shelf or the


Figure 1: Representation of the world in a FPS
type game with the red lines being a path.

AI in Real Time Strategy (RTS
type) games

Just like in first
person shooters,
type games
must implement
a path
finding s
finding is more important in RTS
type games in that s
ome games require this
module to have a solution for movement of hundreds of units on the map and it needs to
be done in seconds.

Because there are numerous computer
controlled players on t
this module
must also

handle collisions between the units so they don’t run into
each other.

The units themselves are controlled by an event driven engine

where action is taken
based on events

The same rules apply here as they do in first
on shooters. It is also
a good idea to use leaking

Most maps are represented by a rectangular grid

as shown in figure 2
A module is used
to analyze the game map.

This module uses a goal driven engine

which works by taking
the highest ranked

goal and process

it. Smaller sub
goals are created as needed and
are processed until the goal has been fulfilled [3]. This module
has to
analyze the

and a settlement can be built based on the evaluation of the terrain.

The same
module d
ecides when cities should be built and how rei
nforcements should be placed


example which shows the interaction between the event driven engine and the goal
driven engine could be as follows: a building gets blown up by an air strike
. This sparks
he event based engine to give a new goal to the goal based engine to increase air
defenses. The goal based engine responds by moving units that are capable of air defense
into position


Figure 2: Representation of the world in a RTS
type game with

the red lines being a path

AI in Role Playing Games


Artificial intelligence in RPGs has
been scarce. They have mostly been
dependent on random encounters or scripted behavior. The random encounters are
mostly seen in the games wher
e there is a lot of fighting and where moving up in levels is
more important


Scripted behavior has some minor AI involved and is common in games where the story
line is more important


These games can have some fighting as well but it is usually

kept to a min

Some games implement a combination of both scripted behavior and
random encounters

AI in Sports Games

A lot

of the
AI in
sports games
has some sort of cheating
. F
or example, in racing games,
two paths are marked on the track: the
first represents the optimal
path; the
second represents the path used when passing opponents.
Basically it is the same as
stored patterns.
The track

split into sectors and each sector length is calculated.
These sectors are used to buil
d a graph describing the track and to acquire characteristics
of the track in the vehicle’s closest vicinity [2].
In effect
, the computer
driver knows

when to slow down if a curve is near because it knows what sectors are approaching.

The AI system in
racing games must also

analyze the terrain to detect objects in the road

so the computer
driver can go around it or hit it if preferred. There must also be

between the AI system and

the physics module [2]. The physics module
provides informa
tion s
ch as when the car is skidding and the

AI system
needs to

immediately to get the vehicle under control.

Figure 3: Segmentation and driving paths of the track. Blue represents optimal path,
dashed pink represents passing path.

Cheating ca
n be found in other sports games as well.
A lot of times
, the computer
player’s behavior
or move
is decided bef
ore the beginning of its turn.

This is not realistic
AI and should be avoided. The computer
player should evaluate the human
move bef
ore deciding a move of its own.

Popular AI Algorithms Used In Computer Games

By h
aving knowledge of a couple of algorithms, one can design a simple AI system for
simple FPS or RTS
type games. One of the algorithms is the A
Star (A*) algorithm
which is
used for
optimal paths. Another algorithm is the finite state machine
which prepares behavior scenarios for the computer players

. One becoming more
popular due to better technology is artificial neural networks.

The A* Algorithm

A key prob
lem in most games is finding a path from two points on a map
Not only does
the A* algorithm find a path between two points, but it finds the shortest path between
two points.
The computer game wor
ld has thoroughly evaluated the

of path

to this date, the A* algorithm has become a standard in games

where path
finding is
a huge part of the computer
player movement


Two prerequisites are needed for A* to work.
There must

a graph or a map

of the
game world. The map

be divided into squares with each square containing a value.
Each square will have eight neighbors unless it is located on the edge of the map or near
an obstacle. A method also needs to be developed
to estimate th
e distance between two
points. This is
the heuristic of the algorithm. This heuristic must underestimate the
distance to ensure the algorithm will find the shortest path. If it overestimates the
distance it will
slightly run faster and
still find a

but it won’t necessarily be the



How about try all paths

and then choose the shortest? This will work, but it takes time
and resources. This is where A* comes in to play. A* minimizes areas of the map to be
examined by orienting the search towards the target

ranteeing to find the shortest
. This is the main advantage of A*. It is faster

and guarantees to find the shortest,
most optimal path
, and in today’s games, we can’t go with out it.

The algorithm
uses some heuristic to help orient the search toward
s the target. Typically,
the heuristic is
the estimated cost of getting

to the destination or target, that is,
distance from the current point being examined to the target or destination.

The heuristic
must be an underestimate of the actual cost. If

it is an overestimate it may run slightly
faster and a path will still be found, but it doesn’t guarantee that it will be the shortest or
optimal path.

ptimal does not always mean the shortest. Additional factors
need to be taken

such as the t
errain, angle of turns, number of enemies in area, and other
s depending on
the game [2].
Some games may have the algorithm

avoid certain areas
or situations
occurring on the map. For example if there is a battle or firing of a weapon you may
want to

distance from teammates or friendly units.


The algorithm can be

a bit


at first.

there is a basic understanding of a

few concepts the algorithm uses, it is not as trivial. One concept is an
open list
open list contains
the nodes that need to be considered as possible starts for future growth
of the path

. The start node is typically placed into this list

at the beginning
. Now
we have an open list,
we will

need a
closed list
The closed list
contains the n
that have had all of their neighbors added to the open list

Each node in the open and
closed lists has a
G score
. The G score indicates the distance or weight of the path
leading to

the node from the start node

. The
re is also an

H score

ich is

that is used. The H score is like the G score except

it represents the distance
from the
node to the endpoint

The tracing out of the path is not completed
ahead of time so you
cannot really know this score.
As discussed

before, a method is
used to estimate this


Starting out
, the closed list is empty

and the starting point
in the open list. Each node
holds its G score and the node that was used to get to the current node for backtracking


Typically th
ese nodes are referred to as parent and children nodes.
The path needs
to be extended until the target destination is reached. This occurs by
calculating the h
score for all of the nodes in the open list and then
picking the node in the open list with


sum of the G and H scores

. If the open list is empty, then there is no path
from the starting point to the end point. Let’s call
the point


very point that is
to T and not in the closed list gets added

to the open list

5]. I
f it is
in the open
list already then don’t add it again.


G score
s of the new nodes in the open list


G score of T plus the distance between the new node and T

. The previous

or parent

node for the new nodes is T.

node was alre
ady in the open list

that is adjacent to T
check if the new G score is less than the current one and if it is, update it

as well as the
previous pointer, otherwise do nothing

. If the new point is the target point, then the
optimal path is found. T i
s moved to the closed list and the process is started over


Finite State Machines

A finite state machine

is one of the most effective and used methods of programming AI.
At the same time, they are one of the least complicated.
Each object in a gam
e can have
a number of states in its life. For instance, a

can be arming hims
elf, patrolling, or
Objects respond in different ways depending what state they are in.


advantage of this
method allows us to divide the implementation o
f each object’s
behavior into smaller pieces, which makes d
ebugging and extension easier


A finite state machine is a model composed of states, transitions, and actions. States
store information about the past [6]. Transitions indicate a state chang
. Typically a
transition condition needs to be fulfilled to complete the transition [6]. There are four
types of actions: entry action, exit action, input action, and transition action. Entry action
is an action that occurs when entering a state [6].
Exit action is an action that occurs
when exiting a state [6]. Input action is an action that occurs depending on input and
present state [6]. Transition action is an action that occurs when performing a transition

A simple diagram that illustrates

this is shown below in figure 4.

Figure 4: Finite State Machine

Artificial Neural Networks

An artificial neural network is a software representation of the human brain. It works by
receiving input, processing the input,
and communicating an output.
The input is
processed in hidden layers as seen in figure 5.
Successful artificial neural networks must
be trainable, that is, learn to solve problems from examples and use the “acquired
knowledge” to solve unseen problems.

Figure 5: Artificial neural network representation

There was a debate

of how to get
artificial neural networks

implemented into computer
. It has been a trendy topic as of late. It has been said that artificial neura
networks have huge potential in computer games.

Collin McRae Rally 2
was one of the
first games to implement an artificial neural network successfully in 2001.

The trained
neural network

in this game

is responsible for keeping the computer player’s ca
r on the
track while trying to go around the track as fast as possible


The input to the neural
network could be
curvature of the road, distance from the corner,


type of road,

of car
, or the car’s properties

. From receiving this in
put, the neural network

can generate an output which is optimal for the given conditions. This output is
passed to the physical layer module

The neural network’s application is limited in games due to a few problems such as


input for neural networks

Need for re
training the network when changes occur in the game

Complicated theory and debugging

Training the network takes time and is complicated

Choosing the network’s input parameters can be a serious problem. The paramet
ers need
to be chosen in a way that will let the network learn to solve situations that have not

. A general rule of thumb is to have the input data represent as much
information about the game world as possible.

A set of input data is ne

for training the network
Several hundred samples for most
today is

usually requires a significant amount of effort
from the developers.

The last step is to train the neural network. Simultaneous testing should be o
with the training process to ensure that the game doesn’t get too difficult or too easy

in need of further training [2]

Fuzzy logic is often used with neural networks

Fuzzy logic

for more human like


It is usually in
the form:


action. For example,



maintain normal speed or



slow down.

When the
use of
fuzzy logic and neural networks

is successful, the results are
with what can be
achieved by using rules hard
coded into the code

traditional logic



Where would the game world be today if it weren’t for artificial intelligence?
I can
guarantee it wouldn’t be where it is today with out artificial intelligence.

It is not
possible to create a game with out AI in today’s world. With out AI in a game is like
Earth without the sun.

AI in gaming has come a long way since the 1970s.
A*, finite state machines, and
artificial neural networks provide games with
the most human like computer player
Tremendous accomplishments

occurring such as
neural networks


being implemented
nto a game for the first time. We are slowly moving
towards having artificial neural networks
and fuzzy logic

being a standard in games like
the A*
algorithm is now. Luckily for gamers, that day is soon here


Game Artificial Intelligence.

Wikipedia E
cyclopedia. September 7, 2006.

Grzyb, Janusz.
Artificial Intelligence in Games

Software Developer’s Journal. June

] Petersson, Anders.
Artificial Intelligence in Games.

WorldForge Newsletter.

[4] Popovic, Zoran; Martin, Steven; Hertzmann, Aaron;
Grochow, Keith.
Inverse Kinematics.



The Game Programming Wiki. September 15, 2006.

Finite State Machine.

Wikipedia Encyclopedia. November 1, 2006.