The Current State of
in Computer Simulations
D. Ezra Sidran
22C 290: 004
Readings for Research
University of Iowa
Professor Joe Kearney
Table of Contents
Statement of the Problem 5
The Simulation Environment 6
Methods of Research 7
Human-Level AI’s Killer Application Interactive Computer Games. Laird; van Lent
Tricks of the Windows Game Programming Gurus. André LaMothe 10
Game Programming Gems. Edited by Mark DeLoura, 2000 18
Artificial Intelligence in Games: A Review. Sweetser (2002) 22
Qualitative Spatial Interpretation of Course-of-Action Diagrams.
Ferguson; Rasch, Jr; Turmel; Forbus (2000) 23
Enhanced Military Decision Modeling Using a MultiAgent System Approach.
Sokolowski (2003) 24
Pathfinding, Strategic, Tactical, and Terrain Analysis: A Look at Artificial Intelligence
in Real Time Strategy Games. Fisher (2001) 26
Smart Moves: Intelligent Pathfinding. Stout (1996) 27
Tactical Movement Planning for Individual Combatants. Reece; Kraus; Dumanoir
A Bayesian Network Representation of a Submarine Commander. Yu (2003) 31
Strategy and Tactics in Military WarGames: The Role and Potential of Artificial
Intelligence. Veale (1997?)
Lecture I: Introduction to WarGaming
Lecture II: Movement, Terrain and Visibility
Lecture III: Modeling Combat and Predicting Losses
Lecture IV: Modeling Fatigue and Morale
Lecture V: Ancient Masters -- how much algorithmic content can we find in Sun
Tzu's "The Art of War"? 33
Fabulous Graduate Research Topics in CGF, Improving the Army Graduate
Program; Marshall; Argueta; Harris; Law; Rieger (2001) 35
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 2 of 116 Army Digitization Overview for NDIA. Balough (2000) 37
DoD Modeling and Simulation (M&S) Glossary 37
U. S. Army Field Manual FM 101-5: Staff Organization And Operations. 38
U. S. Army Field Manual FM 34-130: Intelligence Preparation of the Battlefield. 38
Computing Machinery And Intelligence; Turing (1950) 38
Do We Really Want Computer Generated Forces That Learn? Petty (2001) 39
Realtime Initialization of Planning and Analysis Simulations Based on C4ISR System
Data. Furness; Isensee; Fitzpatrick (2002) 40
Representing Computer Generated Forces Behaviors Using eXtensible Markup
Language (XML) Techniques. Lacy; Dugone (2001) 41
Moneyball: The Art of Winning an Unfair Game. Michael Lewis 2003. 43
Heterogeneous Agent Operations in JWARS. Burdick; Graf; Huynh; Grell;
MacQueen; Argo; Bross; Prince; Johnson; Blacksten (2003) 45
The Representation and Storage of Military Knowledge to Support the Automated
Production of CGF Command Agents. Allsopp; Harrison; Sheppard (2001) 50
Variability in Human Behavior Modeling for Military Simulations. Wray; Laird (2003)
An Intelligent Action Algorithm for Virtual Human Agents. Undeger; Isler; Ipekkan
Terrain Analysis in Realtime Strategy Games. Pottinger (2001?) 58
An Exploration into Computer Games and Computer Generated Forces. Laird
A Modified Naive Bayes Approach for Autonomous Learning in an Intelligent CGF.
Chia; Williams (2003) 63
Modeling Cooperative, Reactive Behaviors on the Battlefield with Intelligent Agents.
Ioerger, Volz; Yen (2001) 67
A Distributed Intelligent Agent Architecture for Simulating Aggregate-Level
Behavior and Interactions on the Battlefield. Zhang, Biggers, He, Reddy, Sepulvado,
Yen, Ioerger (2001) 67
Modeling Adaptive, Asymmetric Behaviors. Bloom (2003) 71
Developing an Artificial Intelligence Engine. van Lent; Laird (1999) 72
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 3 of 116 Agent Based Toolkit for Intelligent Model Development. Napierski; Aykroyd; Jacobs;
White; Harper (2003) 74
CIANC3: An Agent-Based Intelligent Interface for Future Combat Systems
Command and Control. Wood; Zaientz; Beard; Frederiksen; Huber (2003) 75
Game AI: The State of the Industry, Part One. Woodcock (2000) 78
Game AI: The State Of The Industry, Part Two. Pottinger; Laird (2000) 78
Computer Player AI: The New Age. Pottinger (2003) in July 2003 78
Sketching for Military Courses of Action Diagrams. Forbus; Usher; Chapman (2003)
Calculi for Qualitative Spatial Reasoning; Cohn (1996) 82
Describing Topological Relations With Voronoi-Based 9-Intersection Model. Chen;
Li; Li; Gold (1998) 82
How Qualitative Spatial Reasoning Can Improve Strategy Game AIs. Forbus;
Mahoney; Dill (2000) 82
Potential of Qualitative Spatial Reasoning in Strategy Games. Chew (2002) 82
Commander Behavior and Course of Action Selection in JWARS. Vakas; Prince;
Blacksten; Burdick (2001) 85
PC Games: A Testbed for Human Behavior Representation Research and
Evaluation. Biddle; Stretton; Burns (2003) 89
A Method for Incorporating Cultural Effects into a Synthetic Battlespace. Mui;
LaVine; Bagnall; Sargent (2003) 90
A Methodology for Modeling and Representing Expert Knowledge that Supports
Teaching-Based Intelligent Agent Development. Bowman; Tecuci; Boicu (2000) 95
JWARS OUTPUT ANALYSIS; Blacksten; Jones; Poumade; Osborne; Stone (2001)97
AI Middleware: Getting into Character; Part 1 - BioGraphic Technologies'
AI.implant; Dybsand (2003) 102
AI Middleware: Getting into Character; Part 2 - MASA's DirectIA; Dybsand (2003)
AI Middleware: Getting into Character; Part 3 - Criterion's Renderware AI 1.0;
Dybsand (2003) 102
Other Papers of Interest 107
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 4 of 116
When we talk about Artificial Intelligence in the context of strategy games or
military simulations we can all picture what we’re talking about even if we can’t
agree on the methods used to create it or even a precise name to call it.
John Laird has used the phrase “Human-Level Intelligence” by which he means an AI
that is able to compete with a human on an equal level.
George and Cardullo use the term “humanlike expertise in the military domain.”
Grant’s Vicksburg campaign (top) and
MacArthur’s landing at Inchon (right). These
campaigns are classics of military strategy because
the maneuvers were completely unsuspected and
were contradictory to established military doctrine. That is also why they were successful.
However, we know what we want the AI to do: confronted with the situation that
faced Grant before Vicksburg in 1863 we want the AI to ignore conventional
military wisdom and send its troops below the objective and conduct a lightening
“Intelligent systems such as CGF must possess humanlike expertise in the military domain. Like
a human or group of humans in a military organization they must be able to adapt and learn” - G.
R. George and F. Cardullo. “Application of Neuro-Fuzzy Systems to Behavioral Representation in
Computer Generated Forces”, Proceedings of the Eighth Conference on Computer Generated Forces and
Behavioral Representation, Orlando FL, May 11-13 1999, pp. 575-585.
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 5 of 116 campaign that ends with the envelopment and surrender of the enemy forces.
Likewise, in a simulation of the Korean peninsula in September, 1950, we want the
AI to conceive of a daring amphibious invasion 180 miles behind enemy lines as
We want an AI that understands the rules and the physics of its environment and
yet can think outside of the box when necessary.
One of the books making quite a stir this summer is Michael Lewis’ Moneyball. The
book describes the management style of Billy Beane of the Oakland A’s but that is
not what is controversial. Rather it is the book’s description of Bill James’
Sabermetrics and how it throws conventional baseball wisdom out the window that
is causing so much discussion around the ball fields of America. Bill James makes a
very convincing case that 150 years of “common sense” baseball is completely
wrong. Billy Beane, embracing the tenets of Sabermetrics has built a winning ball
club for a fraction of the cost of his opponents. We want an AI that can come to
the same counterintuitive, but correct, conclusions.
What military and sporting simulations have in common is that they are not
Discrete State Games.
Chess is a Discrete State Game . There are a finite number of possible positions ; it
is an extraordinarily large number but it is a definitive number. If one had a
computer, or a book or a file cabinet that was capable of storing every conceivable
position in chess with the best possible move to make at every discrete state in the
game then you would have the ultimate, unbeatable chess master. Call it AI, call it
an expert system, or just call it a giant look up table but it would be definitive and
Also sometimes called an NxM game. “NxM game: A normal form game for two players, where
one player has N possible actions and the other one has M possible actions. In such a game, the
payoffs pairs to any strategy combination can be neatly arranged in a matrix, and the game is easily
analyzable.” See http://www.xs4all.nl/~mgsch/gaming/theory_glossary.htm.
The number is frequently given as .
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 6 of 116
Indeed, any Discrete State Game strategy or Discrete State problem can and will be
solved. It does not matter what methods are used in the short term — neural nets,
genetic algorithms, or brute force — there is a definitive solution waiting out there.
This is not the case of simulations. Computer simulations, by definition, are
emulations of the real world. The real world rarely rests in discrete states.
The purpose of this research is to collect, catalog and summarize published and
available papers written within the last few years on the subject of human-level
artificial intelligence for computer simulations and wargames.
Work in this area falls into three distinct categories:
1. Research done in academia that is published in ACM, IEEE or similar journals.
2. Research done under contract to the military and published in symposium
proceedings such as Behavior Representation in Modeling and Simulation
(BRIMS), the Defense Modeling and Simulation Office (DMSO) Journal or
3. AI routines created for specific commercial computer games that are
presented at industry conferences such as the Game Developers Conference
(GDC) or appear as articles in industry specific books such as LaMothe’s
Game Programming Gems or Laramee’s, AI Game Programming Wisdom.
A conclusion appears at the end of the next section.
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 7 of 116 Human-Level AI’s Killer Application Interactive Computer
Games. Laird; van Lent (2001)
Type of Research:
Academic. This article appeared in the summer 2001 issue of AI Magazine.
Also on this disk: John Laird’s PowerPoint™ presentation “Toward Human-level AI
for Computer Games.”
Laird’s article is practically a call to arms for the AI community. It is frequently cited
in papers that touch on AI and the computer game industry and begins:
“Although one of the fundamental goals of AI is to understand and develop
intelligent systems that have all the capabilities of humans, there is little active
research directly pursuing this goal. We propose that AI for interactive computer
games is an emerging application area in which this goal of human-level AI can
successfully be pursued.”
“The thesis of this article is that interactive computer games are the killer
application for human-level AI. They are the application that will need human-level
AI. Moreover, they can provide the environments for research on the right kinds of
problem that lead to the type of incremental and integrative research needed to
achieve human-level AI.
Laird discusses the advent of Computer Generated Forces (CGF) in 1991 and
describes how he had hoped that this would be, “the right application for our
research that requires the breadth, depth and flexibility of human intelligence.”
However, by late 1997, “we started to look for another application area, one where
could use what we learned from computer-generated forces and pursue further
research on human-level intelligence. We think we have found it in interactive
Laird then proceeds to enumerate the basic computer game genres (Action Games,
Role-Playing Games, Adventure Games, Strategy Games, God Games, Team Sports
and Individual Sports) and describe the AI challenges and opportunities in each
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 8 of 116 In his conclusion, Laird writes:
One attractive aspect of working in computer games is that there is no need to
attempt a “Manhattan Project” approach with a monolithic project that attempts to
create human-level intelligence all at once. Computer games provide an
environment for continual, steady advancement and a series of increasingly difficult
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 9 of 116 Tricks of the Windows Game Programming Gurus. André
LaMothe New York, Sams Publishing 1999.
Book, 1005 pages plus CD-ROM of example source code.
This book is not so much a ‘bible’ of the computer game industry as it is a
grimoire . It is especially loved because it contains numerous source code examples
both in the text and on the accompanying CD-ROM.
Chapter 12, “Making Silicon Think with Artificial Intelligence” does a fine job of
introducing, discussing and showing with source code the most common techniques
of AI for computer games. These include, in order:
These are predetermined or preprogrammed behaviors (including random vectors
and velocities). All external input is ignored. Pac Man is a classic example of
These can be as simple as the standard
if (player_x > monster_x)
or as complicated as curved trajectory tracking and calculating future target location
and adjusting vector and velocity accordingly.
These are simply the reverse of the tracking algorithm above.
Patterns and Basic Control Scripting
Patterns are frequently stored as defines or bits and use a simple vocabulary such
as: GO_FORWARD = 1, TURN_RIGHT_90 = 2, WAIT_X = wait X cycles, etc.
These commands are then read and interpreted though a series of switch
Patterns with Conditional Logic Processing
This combines pre-designed patterns with some conditional such as:
grimoire (noun) - 1. a manual of black magic (for invoking spirits and demons). From
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 10 of 116 If (test_distance > 7)
AI_State = TRACK_PLAYER;
AI_State = AVOID_PLAYER;
Modeling Behavioral State Systems
“To create a truly robust Finite State Machine (FSM) you need two properties:
1. A reasonable number of states, each of which represents a different goal or
2. Lots of input to the FSM, such as the state of the environment and the other
objects within the environment.” — p. 729
A master Finite State Machine (FSM) with substates (from page p. 730).
Elementary State Machines
This involves separating the AI into “High Level” states, “Medium level logic” and
“Low level logic.” See next diagram (from page 731).
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 11 of 116
LaMothe continues stating that, “A personality is nothing more than a collection of
predictable behaviors.” And later, “…you should be able to model personality types
using logic and probability distributions that track a few behavioral traits and place a
probability on each.”
LaMothe also suggests adding a “radii of influence” that further adjusts and
influences a computer personality (see below):
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 12 of 116
Modeling Memory and Learning with Software
This is a fairly obvious AI routine with the addition of agents transferring knowledge
and data. See the sample source code for memory and learning. (Note: Word may
not correctly open the file which is a Visual Studio C++ file.)
Planning and Decision Trees
LaMothe states that high-level AI = planning = a high-level set of actions. He also
suggests building a decision tree of nodes that represent a production rule and/or
an action which can be loaded at runtime from a file.
LaMothe begins by describing basic, primitive pathfinding algorithms including “Trial
This method involves constantly shooting a ‘line’ from the subject to the goal.
Whenever an obstacle is encountered a new path is traced around it:
As LeMothe says, “This works, but it looks a little stupid because the algorithm
traces around things instead of going through the obvious shortest path. But it
works.” But there’s something to be said for an algorithm that is guaranteed to
Collision Avoidance Tracks
This method involves calculating (possibly pre-computing), or ‘drawing’ during level
design, avoidance paths around all obstacles and storing them for later use. See
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 13 of 116
This system is fine if you have the luxury of pre-computing or pre-designing terrain.
This method consists of constructing a series of predefined waypoints and paths
(see diagram below). Source code is also included showing implementation of this
method in a driving/racing game.
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 14 of 116
Breadth-First, Bidirectional Breadth-First, Depth-First, Dijkstra’s Search and A*
Searches are described. However, LaMothe cautions that none of these are
applicable to real-time pathfinding.
Advanced AI Scripting
LaMothe describes some basic scripting techniques and suggests using the compiler
pre-processor commands to turn a pseudo-English scripting language into standard
C/C++ code. The script file (xxx.scr) would be added as an include to the code and
then compiled and linked into the executable.
Artificial Neural Networks
Neural Nets have been around since they were first described by McCulloch and
Pitts in 1943. A neural net basically emulates the human brain by creating logic
circuits that fire when two inputs are greater than a predetermined value (theta). A
bias can also be added to simulate long term memory. See LaMothe’s in depth
article here (neural.doc). Also in the same subdirectory is source code and
executable examples of neural network implementation.
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 15 of 116 Genetic Algorithms
LaMothe only briefly describes Genetic Algorithms that emulate the process of
“natural selection” to solve AI tasks. Obviously, the key to creating robust genetic
algorithms rests with the algorithm evaluation criteria.
Fuzzy Set Theory, Fuzzy Linguistic Variables and Rules, Fuzzy Manifolds and
Membership, Fuzzy Associative Matrices
Key Fuzzy Logic terms and concepts:
DOM = Degree of Membership
FLV = Fuzzy Linguistic Variable
FAM = Fuzzy Associative Matrix (Matrices)
Logical AND means minimum of the set
Logical OR means maximum of the set
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 16 of 116
The key principles of fuzzy logic are:
1. As opposed to “crisp” logic, fuzzy linguistic variables (FLV) do not fall into precise
categories rather they have Degrees of Membership (DOM). See first chart 12.46
2. Sets of FLVs are ANDED (means minimum of the set) or ORED (means
maximum of the set) and then used to index the Fuzzy Associative Matrix (FAM) as
in above (chart 12.47). Each cell of the FAM can have a number of values or
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 17 of 116 Game Programming Gems. Edited by Mark DeLoura, Rockland,
Massachusetts, Charles River Media 2000.
Book, 614 pages plus CD-ROM of example source code.
This book is another “classic” of the computer game industry and, like LaMothe’s
Tricks of the Windows Game Programming Gurus above, includes a CD-ROM of
example source code. Gems, however, is comprised of a series of articles both
written specifically for this book and those reprinted from other sources.
Chapter 3, Artificial Intelligence, includes the following articles:
Designing a General Robust AI Engine by Steve Rabin.
This is a general overview of AI engines used in games and suggests a method as
described in the flow chart below.
A central message router sorts and clears ‘game objects’ that are then sent to a
specific State Machine for processing. Rabin’s message consists of five fields: a
descriptive name, name of sender, name of receiver, time it should be delivered and
relevant data. An example message from the article is:
name:damaged, from:dragon, to:knight, deliver_at_time:245.34, data:10 (damage)
Rabin also states the value of this system because it leaves an ‘electronic paper trail’
which is useful for debugging and the desirability of sending “delayed messages”
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 18 of 116 which can be acted upon at a later time. Caveat: ensure that the game object has
not been destroyed before a delayed message is acted upon.
A Finite-State Machine Class by Eric Dybsand.
This article is primarily source code for implementing an FSM in C++.
Game Trees by Jan Svarovsky.
Game trees are only applicable to discrete games such as chess and checkers and,
therefore, are not of
The Basics of A* for Path Planning by Bryan Stout. Note: Stout also wrote Smart
Moves: Intelligent Pathfinding. Stout (1996) below.
Theoretically an A* search will eventually return the “cheapest” path (as defined by
the code which can weight various terrain types in different “costs”). However, on
large maps an A* search can create thousands of nodes which can quickly consume
all available memory.
Stout writes, “The case in which A* is most inefficient is in determining that no path
is possible between the start and goal locations; in that case, it examines every
possible location accessible from the start before determining that the goal is not
See also Amit's Thoughts on Path-Finding on this disk.
A* Aesthetic Optimizations by Steve Rabin.
This is an interesting short article describing various methods of smoothing A*
paths by using Catmull-Rom and other spline techniques.
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 19 of 116 A* Speed Optimizations by Steve Rabin.
A* is a very slow algorithm because it must search every possible ‘square’ in a map.
For example a 1,000 x 1,000 grid map will produce 1 million squares that need to
This article presents two basic methods for speeding up A* path searches. The first
is reducing the number of ‘squares’.
The diagrams above represent: a) a rectangular (or hexagonal) grid; b) an actual
polygonal floor; c) a polygonal floor representation and d) points of visibility. ‘A’ is,
of course, the slowest. ‘B’ uses the actual polygons created by a 3D program; which
can be extraordinarily complex in many cases. ‘C’ cheats by having a pathing map
created and overlaying the real map. This method is completely unacceptable for
terrains that are not predesigned. ‘D’, again, represents another predesigned
Other techniques suggested are a hierarchical pathfinding system which, again, is
restricted to certain medthods of terrain and/or obstacles representation. Lastly,
modifications of the heuristic cost of the A* search will produce optimatization but
it is impossible to predict at what cost of the quality of the path found.
Simplified 3D Movement and Pathfinding Using Navigation Meshes by Greg Snook.
This method involves “cheating” and precalculated paths which are not applicable to
my are of interest.
Flocking: A Simple Technique for Simulating Group Behavior by Steven Woodcock.
The boids.c source code is included here.
Fuzzy Logic for Video Games by Mason McCuskey.
A very good, but very short, primer for fuzzy logic.
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 20 of 116 A Neural-Net Primer by André LaMothe.
This appears to be an expanded article of what first appeared in his Tricks of the
Windows Game Programming Gurus (see above). This also includes source code on
the CD-ROM and briefly describes Hebian nets and Hopfield autoassociation nets.
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 21 of 116 Artificial Intelligence in Games: A Review. Sweetser (2002)
Type of Research: Academic.
This is an extremely detailed (62 pp.) overview AI techniques currently used in
commercial games. The topics include:
• Finite State Machines
• Fuzzy Logic
• Fuzzy State Machines
• Decision Trees
• Neural Networks
• Genetic Algorithms
• Extensible AI
• Middleware and Research Centres
Each chapter includes an explanation of the technique and examples of current
commercial games that use the technique.
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 22 of 116 Qualitative Spatial Interpretation of Course-of-Action Diagrams.
Ferguson; Rasch, Jr; Turmel; Forbus (2000)
Type of Research:
Academic research funded by Defense Advanced Research Projects Agency
This paper is a precursor of work done by Ferguson et. al. (see Sketching for
Military Courses of Action Diagrams. Forbus; Usher; Chapman (2003)) that
summarizes the work by the Qualitative Reasoning Group of Northwestern
University that was funded by DARPA. This project was involved with the
“digitization” (in the words of Ken Forbus) of certain staff functions performed by
the G2 and S2 officer of a division or corps (see U. S. Army Field Manual Fm 101-5:
Staff Organization and Operations).
A typical Course of Action (COA)
diagram taken from Qualitative
Spatial Interpretation of Course-of-
Action Diagrams. Caption: Figure 4:
Example from a Course-of-Action
diagram. Three friendly brigade-level
task forces attack three enemy
The main attack is against objective
Buford, and the supporting attack is
against objective Grant.
Ferguson et. al. constructed two COA diagram interpreters using their qualitative
spatial reasoning engine, GeoRep (see GeoRep: A Flexible Tool for Spatial
Representation of Line Drawings. Ferguson; Forbus (2000)). The first system used
GeoRep to interpret individual COA glyphs (unit icons and movement orders). The
second system then used GeoRep to describe geographic relationships implied by
the symbol arrangements. This allows the program to respond to questions such as:
What coa-object is located on/at coa-area? Or what is ordinal direction of coa-obj1
relative to coa-obj2?
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 23 of 116 Enhanced Military Decision Modeling Using a MultiAgent System
Approach. Sokolowski (2003)
Type of Research:
Government funded (Joint Warfighting Center, U.S. Joint Forces Command &
Defense Modeling and Simulation Office) academic research.
The author (Sokolowski) is associated with the Virginia Modeling, Analysis &
Simulation Center at Old Dominion University. In this paper he describes the
methods he used to create a multiple agent system, employing Recognition-Primed
Decision Making (RPD), entitled RPDAgent that was designed to simulate strategic
decisions made by a senior military commander. The author first examined von
Neumann’s classical decision theory before deciding upon RPD (a method by Klein
to describe Naturalistic Decision Making (NDM)).
The ‘test bed’ that Sokolowski created was a hypothetical amphibious landing.
Sokolowski then presented the scenario to thirty senior U. S. and Coalition military
officers and recorded their responses to four strategic decision making points:
1. Which of four beaches to attack?
2. When to attack?
3. Should the timing of the attack be changed due to enemy troop movements?
4. Should the attack be called off after encountering high casualties and strong
RPDAgent evaluated 20 variables that were narrowly defined such as Beach
Topography: Sand Type: Coarse or Fine (see also Representing Knowledge and
Experience in RPDAgent. Sokolowski (2003)). RPDAgent, which also employed
stochastic methods, was run 200 times and its responses to the four above
decisions were recorded. The RPDAgent decisions were remarkably similar those
of the 30 senior military officers (showing a standard deviation of between 0 and
Sokolowski then conducted a version of the famous Turing Test (see Computing
Machinery and Intelligence) by showing the results of the RPDAgent simulation runs
Klein, G. Sources of Power: How People Make Decisions. The MIT Press, Cambridge, MA, 1998
and Klein, G. "Strategies of Decision Making," Military Review, Vol. 69, No. 5, pp. 56-64,
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 24 of 116 combined with the responses of the original 30 senior military officers to five
generals (three four-star and two three-star generals) and recorded their
The results (shown below) would qualify for “passing” the Turing Test.
Num. of can’t correct human
SME human or model Percent correct
tell or computer
Gen . A 20 0 0 0
Gen. B 6 14 8 40%
Gen. C 13 7 3 15%
Gen. D 18 2 2 10%
Gen. E 17 3 2 10%
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 25 of 116 Pathfinding, Strategic, Tactical, and Terrain Analysis: A Look at
Artificial Intelligence in Real Time Strategy Games. Fisher (2001)
Type of Research:
Academic; student research.
This article is a very broad overview of pathfinding techniques currently being used
in commercial Real Time Strategy (RTS) games. The author points out that “Most
games today use some variation of the A* algorithm… This algorithm is based on a
heuristic function that ranks nodes in a possible path by… an estimate of the best
path through that node.” This algorithm and numerous variations of it are covered
in greater detail in Smart Moves: Intelligent Pathfinding. Stout (1996)
The author also made observations about the implementation of pathfinding
algorithms in two classic RTSs: Warcraft and Age of Empires 2 and their ability to
“look ahead.” Fisher writes, “…[G]ains are readily apparent when comparing
Warcraft (the oldest game in the pack) with Age of Empires 2 (the newest). The Age
of Empires units are better able to determine a "good" route to a goal. In Warcraft,
the units would not reevaluate their positions when moving around objects. They
would treat each new obstacle as a separate entity and move around it until
reaching the original vector of their path. This wastes time and is an obvious flaw in
the pathfinding. Also, the Warcraft units are slower to react (even on the same test
machine as the Age units), slower to calculate an alternative when faced with an
obstacle, and less likely to take a "good" route when there is unit congestion (a
clustering of units, or traffic jam). Furthermore, there is more unit congestion
overall as the units are less able to detangle themselves from other units.”
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 26 of 116 Smart Moves: Intelligent Pathfinding. Stout (1996)
Type of Research: Computer Game.
Stout has created a test bed computer program (screen shot below) that allows for
the testing of various Pathing techniques.
These include bidirectional breadth-first, Dijkstra’s algorithm, depth-first, Iterative-
deepening depth-first search, Best-first search and A* search algorithms. Each are
displayed in his test bed.
This is a superb starting point for examining various search algorithms. In addition
Stout discusses generalized cylinders, convex polys and quadtrees. This paper
includes numerous graphics and code samples.
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 27 of 116 Tactical Movement Planning for Individual Combatants. Reece;
Kraus; Dumanoir (2001)
Type of Research:
Military; Private corporation, Science Applications International Corporation
(SAIC), funded by U.S. Army’s Simulation, Training and Instrumentation Command
This research was undertaken to deal with problems encountered by SAIC with
their Dismounted Infantry Semi-Automated Forces (DISAF) project (for more
information see this web page). Specifically, SAIC’s Modular Semi-Automated Forces
(ModSAF) software proved inadequate to the task of moving individual soldiers
rather than vehicles.
This paper describes a multi-step process for pathfinding. The first step uses four
levels of decision making to set objective points:
1. Long distance or strategic planning (which is ignored here).
2. Intermediate distance of “a few hundred meters”.
3. Short distance of “about a hundred meters”.
4. “Fine motion” which presumably covers everything less than a 100 meters.
However, the details of setting the objective points are not discussed in this paper.
The paper then discusses four different pathfinding algorithms:
1. Cell decomposition.
3. Weighted regions.
4. Potential functions.
Cell decomposition consists of casting a grid across the terrain and then using an A*
algorithm to assess potential paths. Each cell in the grid — which can be rectangular,
hexagonal or irregular — may be given weighted values representing “anything
related to movement, e.g., trafficablity, ground slope, or exposure to threats”.
The skeleton method employs Voronoi diagrams (see How Qualitative Spatial
Reasoning Can Improve Strategy Game Ais. Forbus; Mahoney; Dill (2000) and
Describing Topological Relations With Voronoi-Based 9-Intersection Model. Chen;
Li; Li; Gold (1998)
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 28 of 116 for a description of Voronoi diagrams as applied to terrain pathfinding).
The ‘weighted regions’ method was first described in “A Stochastic Approach to the
Weighted-Region Problem: Design and Testing of a Path Annealing Algorithm” and
employs a method of first assigning values to each cell and then casting rays in all
directions, “… from the start point; each ray bends at region boundaries to obey
the optimality criterion... The goal point must be “trapped” between two rays. The
path found will be closer to optimal if more rays are used.”
The ‘potential method’ of pathfinding constructs, “a scalar potential (as in energy)
function that steadily decreases to a minimum as the distance to the goal decreases.
The potential function is high at obstacle boundaries and decreases as the distance
to the obstacle increases. The path is determined by following the steepest descent
down the potential value surface until the goal is reached.”
The paper then evaluated each algorithm in relation to the following criteria:
1. Path planning through an obstacle field
2. Planning an optimal path through regions of different cost.
3. Planning an optimal path through terrain with continuously varying cost.
4. Using exposure to threats as a terrain cost.
The table below summarizes the strengths and weaknesses of each algorithm as
indexed with the evaluation criteria:
Cell decomp. Skeleton Weighted Potential Fn.
Fair Good Good Fair
Varying costs Good Poor Poor
Continuous Poor Poor Poor
Threats Good Poor Poor Poor
The authors conclude, “From the table, it is apparent that there is no approach that
is good for all the requirements, but the cell decomposition approach is at least fair
for all. We have selected cell decomposition as the path planning mechanism.”
The authors then construct a ‘search space’ of a “grid of 1 meter squares no more
than 250 meters long by 200 meters wide,” that is oriented from the starting point
to the objective point. The grid is then populated with polygonal terrain obstacles.
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 29 of 116 An A* algorithm is then employed to determine a path. Each cell is weighted based
on ground slope and exposure to enemy threats.
When a path is returned by the A* algorithm it is then ‘post-processed with the
1. Exposed segments of the path are grown by one node so that the agent will have
a chance to accelerate to a rushing speed before being exposed.
2. Segments requiring the agent to be prone are grown by one node so that the
agent will be prone before it enters the area where it needs to be prone.
3. Standing segments between prone segments are checked to see how long they
are. Short standing segments are replaced by prone segments so that the agent
does not waste time standing up for just a short segment. If the standing segment is
exposed, the distance threshold is increased so that the agent will more likely stand
4. The path is examined to see if uniform segments–segments of the path that
don’t change direction more than 45 degrees,
Left: Examples of Path planned through building to avoid exposure to threat (star).
Right: Path planned up a steep hill. The path goes laterally and then traverses the
hill at an angle so that the effective slope is reduced (right).
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 30 of 116 A Bayesian Network Representation of a Submarine
Commander. Yu (2003)
Type of Research:
Military; Private corporation, Frontier Technology, Inc. (FTI), funded by U. S.
Defense Modeling and Simulation Office (DMSO).
This paper describes FTI’s efforts to create a Human Behavior Representation
(HBR) as part of a DMSO Challenge Problem to simulate a human submarine
commander’s decision making processes during a “Sea Scenario” test. “The object
of the “game” was to command a submarine to protect a “high-value asset” (HVA-
an aircraft carrier) and two picket ships from submarine attacks while not
accidentally attacking a non-aggressive submarine.”
The only commands that the HBR could give were:
1) Station on Port - positions the submarine in a location to monitor the enemy
2) Transit- moves the submarine to the specified location at the specified speed
3) Station on Unit- positions the submarine in a location to monitor either the HVA
or one of the picket ships
4) Station in Area- positions the submarine at a specified location
5) Trail- follow a specified enemy sub
6) Disengage Trail- stop following an enemy sub
7) Fire- launch a torpedo against an enemy sub
8) Report- send an information report to HQ
9) Cease Reporting- stop sending an information report
10) Set Speed- set the speed of motion
Furthermore, the HBR did not concern itself with the intricacies of submarine
command such as changes in depth or changing course to properly align the boat
for a torpedo attack.
Yu’s group decided to implement a “belief-based” Bayesian network for the HBR. It
was driven by the following beliefs:
- Belief that Red will attack the HVA (or Picket ship)
- Belief that our submarine is capable of successful attack on Red
- Belief in the location of Red
- Belief that Red is trying to attack our submarine
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 31 of 116 - Belief in what our optimal speed should be
- Belief that we need to monitor the port area
- Belief that we should make a report
- Belief in our best course of action (based on all of the above)
They then purchased Netica, a Commercial-Off-The-Shelf (COTS) software package
that implemented Bayesian Networks (see their web site at
http://www.norsys.com/). The Bayesian Network evaluated a number of
“Conditional Probabilities” They also created four different files that represented
personalities of commanders (aggressive, conservative, skeptical and trusting) that
could be loaded at runtime.
The paper concludes that, “The behaviors witnessed by the test monitors were
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 32 of 116 Strategy and Tactics in Military WarGames: The Role and
Potential of Artificial Intelligence. Veale (1997?)
Lecture I: Introduction to WarGaming
Lecture II: Movement, Terrain and Visibility
Lecture III: Modeling Combat and Predicting Losses
Lecture IV: Modeling Fatigue and Morale
Lecture V: Ancient Masters -- how much algorithmic content can we find in Sun
Tzu's "The Art of War"?
Type of Research:
Academic. This is a series of lectures from a course in Artificial Intelligence and
Games taught at Dublin City University, Dublin, Ireland by Tony Veale.
The author covers basic wargaming and military terms and history including spatial
resolution, facing, early board wargames, kriegspiel, imperfect information (fog of
war) and Line of Sight (LOS).
Battle of Cannae, 216 BC.
Roman Legions and Allies
Also covered are formulas for observation probabilities, the Lanchaster Attrition
Model, Combat Cycles and Combat Efficiency Coefficients. Also included and cited
are combat resolution tables from this researcher’s commercial computer wargame,
The War College (published by GameTek 1995).
This is a very good overview of the history of wargaming over the last 200 years
and is cited in a number of other papers.
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 33 of 116 Fabulous Graduate Research Topics in CGF, Improving the Army
Graduate Program; Marshall; Argueta; Harris; Law; Rieger (2001)
Type of Research:
The intended audience for this paper is Army officers who are attending graduate
school (primarily the Simulation Masters program at the University of Central
Florida) with the hope of directing their research towards topics that are
immediately applicable to specific needs of STRICOM (Simulation, Training and
The 17 specific topics are:
Weapon & Ammunitions Prioritization: a system that automatically determines the
write weapon and ammunition for the target rather than depending on human
Target Prioritization & Disengagement: “Given an array of targets in an intense
battle, one of the most difficult issues a crew has is which target it should engage.”
User Control Interface (UCI)/Workstation & Execution Matrices Improvements:
Not only how information is displayed but how low-level units interact and how
information is shared between them.
Maneuver/Obstacle Avoidance/ Routing Improvement: “One of the most common
complaints from SAF (Semi Automated Forces) operators is the maneuver capability
of the SAF entities.”
Suppressive Fire Effects: “One weakness in current virtual simulations and CGFs is a
realistic simulation of suppressive fire effects.”
Stability and Support Operations (SASO), Nuclear Biological Chemical (NBC), or
Amphibious Operations Requirement Analysis: These new units types are not
accurately modeled or included in CGF simulations.
Virtual Warrior/ Infantry Simulation: “Most of the simulators and CGFs were based
on supporting tank simulation. As the virtual and CGF technology progressed it
became apparent that infantry had to be added to the simulation. The addition of
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 34 of 116 infantry has proven to be difficult because of it unique behavior and maneuver
Military Operations in Urban Terrain (MOUT): There are numerous problems with
urban combat simulation including Line of Sight (LOS), 3D maneuvering, changing
terrain (building damage), etc., etc.
Situational Behavior Improvements: The CGF AI needs a great deal of improvement.
Digital Message Processing: Creating the ability for the CGF to send and receive and
respond to messages.
Fair Fight with Manned Simulators: Manned simulators and SAF are not “equal” in
Executive Framework Control: Optimization of ModSAF and CCTT SAF which has
some definite time and size problems.
Demand-Driven Simulation: Optimization of training simulations so that they would
not spend CPU time and resources calculating unnecessary data.
Scalability Analysis of Simulation Problems and Algorithms: “This analysis could
characterize the potential for time-space tradeoffs in algorithms [e.g. Line-of-Sight
determination, Route Planning] for solving these problems, determine the extent to
which these problems can be solved via parallel or distributed algorithms, and
describe the effects of bandwidth constraints and network latency on parallel or
Command from Simulator (CFS) Issues: In order to “support Command Force
Exercises (CFX) whose focus is the training and exercise of commanders and
leaders, an operational mode called CFS was created in CCTT SAF. In other SAF’S
it has been refereed to as a tethered unit.” Apparently this kludge has caused
numerous problems including units not stopping “on an intervisability line such as a
Aggregate-Disaggregate Algorithms: “For example, the destruction of one tank in a
platoon is not the same as minor damage to two of the tanks, but commonly used
reaggregation techniques might map these back to the same constructive
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 35 of 116 Opposing Forces (OPFOR) Skill Levels/Variable Skill Leadership Behaviors:
“Current SAF systems have methods to create different skill levels that modify
detection, engagement rates and delivery accuracy based on the competency level
identified. This implementation(s) are mainly speculative and need an empirical
analysis based on actual field data.”
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 36 of 116 Army Digitization Overview for NDIA. Balough (2000)
Type of Research: Military.
This is a PowerPoint slide presentation that is the blueprint for the complete
“digitization” of the U.S. Army. Digitization includes not only computer military
simulations but making every unit and CCCI as digital as possible.
Secretary of Defense in a real-time conference with Qatar War Room; part of the
Army’s digitization plan in operation.
DoD Modeling and Simulation (M&S) Glossary
Type of Research: Military.
Definitions and glossary for a blizzard of acronyms. I’ve found this 185 page
document indispensable in deciphering some papers.
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 37 of 116 U. S. Army Field Manual FM 101-5: Staff Organization And
Type of Research: Military.
These are the step by step instructions to create Courses of Action (COAs). Any
computer program that would attempt to simulate the COA procedure would have
to use this manual as a guide.
U. S. Army Field Manual FM 34-130: Intelligence Preparation of
Type of Research: Military.
This manual was prepared for Intelligence Officers creating COAs. It is to be used
in conjunction with FM 101-5: Staff Organization and Operations (see above).
This particular manual is very well written and includes a number of important
insights into the history of successful strategy and tactics.
Computing Machinery And Intelligence; Turing (1950)
Type of Research: Academic
Turing’s classic paper that introduces the “Turing Test”. I added this, not only
because it is such an important document for me, but because it’s nice to always be
able to find a copy of it when you need it.
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 38 of 116 Do We Really Want Computer Generated Forces That Learn?
Type of Research: Academic.
The main thesis is that we (not surprisingly) do not want CGF that learn (at least
most of the time). The argument can be summed up by the chart (below) that
shows the positives (+) and minuses (-) for learning behavior in each category of
CGF CGF applications
behavior Training Analysis Experimentation
Unchanged 0 0 0
+ Improved behavior + Improved behavior + Improved behavior
Improved + Varying experience - Loss of repeatability + Richer experiment
- Loss of training control - Confounded results - Confounded results
- Cost of learning phase - Cost of learning phase - Cost of learning phase
+ Varying experience - Loss of repeatability + Richer experiment
Altered - Loss of training control - Confounded results - Confounded results
- Non-doctrinal behavior - Non-doctrinal behavior - Non-doctrinal behavior
+ Varying experience - Loss of repeatability + Richer experiment
Martian - Loss of training control - Confounded results - Confounded results
- Unrealistic behavior - Unrealistic behavior - Unrealistic behavior
I would argue that there are some important categories that are omitted from the
above chart such as “Learning Something New”, “Looking Outside the Box” and
“Reevaluating Standard Military Doctrine”.
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 39 of 116 Realtime Initialization of Planning and Analysis Simulations Based
on C4ISR System Data. Furness; Isensee; Fitzpatrick (2002)
Type of Research: Private Corporation and Military
Summary: C4ISR = Command, Control, Communications & Computer Intelligence,
Surveillance & Reconnaissance.
There is a very real problem in that heretofore there was not a consistent data
structure among the numerous DoD systems. Ideally, the data that is displayed on
the computer screens inside command vehicles and in the War Room should be
seamlessly imported into simulation programs. This paper proposes using the GCCS
Ambassador as an interface. “As part of previous C4ISR-Simulation interoperability
experiments, the Naval Research Laboratory (NRL) had already developed a GCCS
HLA-compliant interface (known as the "GCCS Ambassador") that allowed
simulation data to be sent via the RTI and deposited into the GCCS Track Database
Manager (TDBM). This interface was originally developed to work with the Joint
Theater Level Simulation (JTLS), an aggregate-level simulation of air, ground, and sea
operations used primarily by the Joint Warfighting Center (JWFC) in Joint training
“While any simulation that is HLA compliant could in theory derive data from
GCCS (Global Command & Control System) in this manner, there are only a select
few that are used in the execution of COA (Course Of Action) and deliberate
planning applications that really need this capability. The Joint Warfare System
(JWARS) is one of the simulations that is currently considering adopting this
technology for use. As JWARS has a requirement specified in its Operations
Requirements Document (ORD) for operating as a COA application [JWARS,
1998] in support of CINC analysis requirements, this type of initialization scheme
may be well suited for it. JWARS has recently developed an HLA interface for use
with other applications, which would provide a straightforward approach for
implementation with the GCCS Ambassador.”
“While the GCCS COP provides a significant amount of information on enemy and
friendly units, it does not contain several bits of data of interest to the analysis of
COAs. Among the data sources that would be beneficial for inclusion in this
federation are the Master Intelligence Database (MIDB), Status of Resources and
Training Systems (SORTS), and Joint Operation Planning and Execution System
(JOPES) data. Also, since GCCS is an aggregated view of the entire theater battle, it
tends to lack certain detail that can be found in component data sources such as
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 40 of 116 Theater Battle Management Core System (TBMCS), and the Joint Common
Database (JCDB). Implementation of COA analysis at lower echelons may require
finer resolution data that can only be found in these component C4ISR systems.
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 41 of 116 Representing Computer Generated Forces Behaviors Using
eXtensible Markup Language (XML) Techniques. Lacy; Dugone
Type of Research: Private Corporation and Military
Summary: The lack of a standard language and data structure that can transfer data
and CGF behaviors between applications is a constant hindrance in development of
CGF applications (see also Realtime Initialization of Planning and Analysis
Simulations Based on C4ISR System Data. Furness; Isensee; Fitzpatrick (2002)
above). This paper recommends XML as a standard and points out numerous
COTS (Consumer Off The Shelf) XML applications are available.
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 42 of 116 Moneyball: The Art of Winning an Unfair Game. Michael Lewis
New York, W. W. Norton & Company, 2003.
Book, 288 pages.
This may be the most important book — and certainly one of the most influential
books — that I read this summer.
While this book, on the surface, is about Billy Beane, the General Manager of the
Oakland A’s and his successful rebuilding of a team on a shoestring budget, it also
introduced me to the history of “sabermetrics” (also “sabremetrics” and
“sabrmetrics”) that Beane used as his manifesto. Sabermetrics (which derives its
name from the Society of American Baseball Research (SABR) was created by Bill
James in the late 1970s (see http://www.baseball1.com/c-sabr.html for a good
overview of the subject).
James, who to the best of my knowledge, was neither a statistician, mathematician
or computer programmer, after pouring over thousands of pages of statistical data
discovered an extraordinarily important, simple truth about baseball that had been
overlooked for over 150 years: outs are bad for the team at bat. Anything that
creates outs was to be eliminated; anything that decreased outs was to be
encouraged. From this simple statement an entirely new concept in baseball was
created that said, “batting averages are irrelevant, RBIs are irrelevant, and the only
thing that mattered was a player’s On Base Percentage (OBP). About 15 years later
Voros McCracken, a disciple of James, discovered the corollary for pitching: the
only things that are actually under the pitcher’s control are strikeouts, walks and
homerun balls; everything else is luck. Consequently, the Earned Run Average (ERA)
Obviously any science (or cult for that matter) that produced thousands of pages of
statistics and theory was going to be a bit more complex than my simple summing
up in the above paragraph but I have captured the heart of Sabermetrics: Outs bad,
OBP important, ERA irrelevant.
There is also one other important fact that I learned from this book: Conventional
baseball wisdom was wrong. This was an epiphany for me.
Previously, if I had been hired by a computer game company to write the AI for a
baseball simulation I would have written it using the complicated (and sometimes
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 43 of 116 contradictory) rules of conventional baseball wisdom that I had been taught since
my youth. For some time I had suspected that many of these conventional baseball
rules were wrong — they had just felt wrong — but I certainly would never have
jumped to the conclusions that James had. However, the results of Sabermetrics are
Now, what intrigues me is this: to write a computer program that crunched all the
data that James did by hand is not a big programming accomplishment (indeed
there’s a nice little C program at http://www.baseball1.com/bb-
data/grabiner/brock2.c that will do that) rather it was the conclusion that James
arrived at that interested me and I asked myself, “Is it possible to write a computer
program that given the rules of baseball and the same data set that James had would
arrive at a similar solution?” Of course, the next logical step if such a program was
created would be to ask: Could an ‘all-purpose’ (universal, generic) version of this
program be written that would:
1. Load in a (game/simulation) rule set.
2. Load in data files (which could include maps, statistics, etc.).
3. Derive an optimum winning strategy.
I can only assume that this is not an original idea.
Internet searches for “optimum strategy reasoning software” return “agent based
systems” and “game theory.”
Internet searches for “universal [strategy] reasoning software” return “case based
reasoning”, “Forbus’s “Qualitative Spatial Reasoning”, “Universal Learning
Machines” and “computational reasoning.”
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 44 of 116 Heterogeneous Agent Operations in JWARS. Burdick; Graf;
Huynh; Grell; MacQueen; Argo; Bross; Prince; Johnson; Blacksten
Type of Research: Private Corporation and Military
This is an overview of how an individual unit in the JWARS wargame is
‘programmed’ (for lack of a better term).
A unit in a JWARS simulation is called a “Battle Space Entity” (BSE).
“Figure 1 (above) shows the general structure of a JWARS Battle Space Entity (BSE),
the primary object for all JWARS behavior. For JWARS land forces, agents are
configured as military units and civilian groups. These units are not decomposable,
but can temporarily spawn subordinate units for specific tasks, if the need arises.”
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 45 of 116
Figure 2 (above) KBs for New Units are Built from Multiple Sources Appropriate
for the Particular Unit
Function: Cbt, CS, or CSS Echelon: Bn, Brigade
Type: Armor, Mech, Inf Is a Headquarters?
Role: Left Flank, Reserve Rank or Skill Level
Unit is under fire Days in combat
Unit is in contact # of enemy in contact
Unit Current Strength Formation/Orientation
Unit Current Objective Local Activity/Mission
Units is On/Off Plan Has specific asset
Is Day/Night Weapons free/tight
Chemical Use Authorized Unit is in Country X
Vegetation type Terrain type
Weather Civilians are present
Figure 3 (above) Primitive Knowledge Base Facts. It is interesting to note that only
28 variables are stored for each unit and does not include such factors as
“leadership”, “morale”, “attack”, “defense” or “speed” which are standards in most
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 46 of 116
Primitive Facts Related Derived Facts
# of Personnel Skill Levels
Type of Equipment Condition/Strength
Amount of Supplies Expected Resupply
Knowledge of Capabilities
Ability to Sense Enemy Situation
Communications Intercept Potential
Own Operations Available Options
Enemy Operations Enemy Intentions
Own Doctrine Expected Enemy Reaction
Weather (current) Favorable for unit
Weather (forecast) Favorable for unit
Terrain Favorable for unit
“Figure 4 (above) Reasoning from Available Facts. Derived facts may only have
applicability to the units triggering them.” These appear to be very simple
extrapolations from the individual BSE data available.
“Figure 5 (above) JWARS Rule Builder User Interface.” This appears to be a simple
Boolean logic / conditional statement editor.
“Without the KB, there have been instances at the lower echelons in JWARS when
combat units that should have engaged one another have missed that opportunity
by simply returning fire and continuing on to their objective. Circumstances have
also occurred where units without KBs have inappropriately engaged in combat. For
example, in some situations a logistics support unit may inappropriately assess the
situation and close with an enemy infantry battalion. The KB supports the desired
outcome by improving the ability to assess the situation and subsequently altering
individual unit courses of action. As shown in Figure 7 (below), attacking combat
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 47 of 116 units sense enemy fire, close with the enemy units, and, after destroying them or
forcing them to withdraw, resume their original mission.”
This diagram labeled Figure 7 (above) was added to the article to illustrate “Altering
Course of Action Using KB Rules” however, I am extremely dubious about the
appropriateness of a unit making an abrupt right angle turn and exposing its left
flank to two hostile units even if it is now attacking a headquarters unit. Indeed this
appears to demonstrate a very common and very old problem with commercial
wargame AIs: units making individual decisions as opposed to an entire army
working as a cohesive whole.
See my revised diagram below:
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 48 of 116
I have indicated with thick green arrows (above) the revised Course of Action
(COA) that certainly follows current doctrine and would probably be the COA that
most (if not all) staff officers would recommend.
JWARS is the most important computer simulation currently used by the Joint
Chiefs and strategic planners for wargaming. It does not have any Artificial
Intelligence capabilities and the introduction of Agents and Knowledge Bases
appears to be the first attempt to do so. In this case the Agents are used to control
subordinate units and not to simulate enemy forces and responses.
It appears that the JWARS Program has encountered many of the same problems
that commercial computer wargames have dealt with for the last twenty years.
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 49 of 116 The Representation and Storage of Military Knowledge to
Support the Automated Production of CGF Command Agents.
Allsopp; Harrison; Sheppard (2001)
Type of Research: Military (United Kingdom funded)
This paper describes the procedures used for creating a central repository for
Knowledge Bases used to program Agents in wargames.
“To derive a highly autonomous CGF agent, the knowledge engineer has to resort
to interviewing a subject matter expert (SME) or to examining source doctrine to
capture the skills needed within the program. The process is iterative and can
involve several SME interviews or several reviews of source doctrine to clarify
issues. The CGF development cycle could be improved by reusing CGF
components from previous applications leading to cost and time savings.”
While I would not have assumed there would have been a great deal of expense
involved in interviewing SMEs and programming the rules, it is certainly a good idea
to create one central set of rules. The architecture for the system now being
developed and used in the United Kingdom looks like this:
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 50 of 116 Variability in Human Behavior Modeling for Military Simulations.
Wray; Laird (2003)
Type of Research: Academic funded by the Office of Naval Research, Virtual
Training & Environments Program,
This article is an overview of the approach that Laird is using to create urban
warfare (MOUT) agents (MOUTBots) with his SOAR language. It does not include
and specific source code and many of the ideas mentioned in the article have not
yet been implemented.
The two basic themes of the article are:
Space of all possible behaviors
(given some initial situation)
Agent 2 (alternate run)
“Figure 1: A notional view of behavior space. As illustrated in Figure 1, within the
space of all correct or “good” behaviors, different [human behavior models] HBMs
(or the same HBM at a different time) can follow different paths through the
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 51 of 116 In other words: when properly implemented two MOUTBots would make different
decisions but still be in the “good” decision category.
rules of engagement
“Figure 3 Variability influences all aspects of HBM behavior generation. …[W]e
introduce a simple model of behavior generation with perception, reasoning, and
action components. Figure 3 provides a notional view of such an agent… Agents
receive input from some external task environment. This input must be
transformed to a representation that the agent can understand. Internally, the
process of perception is mediated by situation interpretation knowledge, allowing
the external situation to be interpreted in terms of the current mission, goals, and
rules of engagement, etc. Reasoning is the process of generating, evaluating, and
selecting goals and actions. Reasoning is influenced by the agent’s current
situational assessment, its background knowledge, emotional state, and current
physical capabilities. The selections of goals lead to actions that are executed in the
Conclusion: This seems like a logical way of creating agents inside an urban warfare
simulation that display certain human characteristics including variability of
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 52 of 116 decisions. However, this article is completely theoretical and abstract. It appears
that so far there has been no attempt to actually create these agents inside of a
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 53 of 116 An Intelligent Action Algorithm for Virtual Human Agents.
Undeger; Isler; Ipekkan (2001)
Type of Research: Academic probably funded by military.
This article presents a method for moving intelligent agents across a 3D landscape
towards a common goal.
Part of this problem, of course, includes determining Line of Sight (LOS) for the
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 54 of 116
The algorithm for individual agent movement is:
For each group
For each agent
If the agent is a red team member (intelligent agent)
Construct the sensor detection list (perceptions) for the agent; > (A)
Analyse the detected list and update the knowledgebase; > (B)
Execute behavior module; > (C)
Update the pysical appearance;
Backup the previous detected list and create a new list;
For each group
For each agent (target) except himself
Calculate the seeing and hearing statistics between the agent and the target; > (D)
For each member of detected list
If the detection exists in the previous list and unsensed because of the probability test and no probability change
occurred after that time
Mark the member of new detected list as “unsensed”;
For each sensed member of new detected list
Do a comparison to knowledgebase, find the similarities;
If similarity found
Update the knowledge and check the member of detected list as “similarity_found”;
For each member of new detected list which is not checked as “similarity_found”
Do a probability test and if it is passed
Add the list member to the knowledgebase as a new perception;
Check the member of detected list as “unsensed”;
Find who the commander is;
Find the status of the mission plan;
If the agent is a commander
If there is no abnormal condition
Follow the path;
Execute the reaction planning module;
If there is no abnormal condition
If the commander position is known
Follow the commander;
Stop and search for the commander;
Execute the reaction planning module;
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 55 of 116 D
Compute the statistics between the agent and the target (line of sight, viewing angle, range, etc.);
If there is any possibility of detection, add the perception to the detected list;
The authors are using a DTED map file and seem to have duplicated much of the
work that my group and I completed in 1995 for The War College (below):
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 56 of 116 They also use an extremely simplified movement matrix (1 movement point when
crossing an embedded graph cells vertically or horizontally and 1.4 movement
points when crossing a cell diagonally). They do not factor in any terrain costs
(woods versus grassland versus swamp, etc.). They do, however, calculate slope and
adjust movement accordingly.
Their algorithm for coordinated agent movement follows:
Description Peusedo Code
Create the world cPlatoon * plt; cAgent * agent; cGoalPoint * point; cGoalItem * item
database. Create the world->Initialize();
DTED matrix world->Create_World(...);
Add a group and two plt = world->Add_Group( GroupRed );
agents to the agent = plt->Add_Agent(...);
agent = plt->Add_Agent(...);
Set group position and plt->SetGroupPosition( 9163.0, 10259.0 );
formation of the agents plt->Set_Agents_Group_Formation(8,true);
Get the starting control point = plt->plt_goals->GetObjectFromIndex(0);
point. Add a goal item point->movingstate = MovingFast;
for the point. Move fast
Goal item Item=point->Add_GoalItem(giWaitUntilReceiveKeyword);
to the next point 1.1 item->data.ReceiveFromID = 1;
item->SetKeyword("First Step Go...");
Add a control point point = plt->Add_Plt_GoalPoint( gpPassThrough, 9345.0, 9742.0 );
Add a tactical control point = plt->Add_Plt_GoalPoint( gpTactical , 9220.0, 9595.0 );
point. Wait until the Goal item item= point ->Add_GoalItem(giWaitUntilContinueMission);
group 2 leaves from
3.1 item->data.ReceiveFromID = 2;
item->data.GoalPointID = 3;
Add another tactical point = plt->Add_Plt_GoalPoint( gpTactical , 8945.0, 9270.0 );
control point. Wait until point ->movingstate = MovingSlow;
group 1 arrives at point
5. When ready to
Goal item item = point ->Add_GoalItem(giWaitUntilArrival);
continue, send a
4.1 item->data.ReceiveFromID = 1;
keyword to group 1. item->data.GoalPointID = 5;
Wait for a response
item = point ->Add_GoalItem(giReadyToContinueDo);
from group 1 to leave
Goal item item->data.doAction = doSendKeyword;
the point. Move slow to
4.2 item->data.SendToID = 1;
the next point
item->SetKeyword("Group 3 Ready");
Goal item item = point ->Add_GoalItem(giWaitUntilResponseKeyword);
4.3 item->data.ReceiveFromID = 1;
Add the target point. Put point = plt->Add_Plt_GoalPoint( gpTarget , 8935.0, 9130.0 );
a bomb to the point and point ->movingstate = MovingFast;
move fast after leaving.
While leaving the point Goal item item = point ->Add_GoalItem(giArrivalDo);
send the keyword 5.1 item->data.doAction = doPutBomb;
Goal item item = point ->Add_GoalItem(giContinueMissionDo);
5.2 item->data.doAction = doSendKeyword;
item->data.SendToID = -1;
Add another control point = plt->Add_Plt_GoalPoint( gpPassThrough, 9212.0, 9124.0 );
Home point point = plt->Add_Plt_GoalPoint( gpHome , 9683.0, 8400.0 );
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 57 of 116 Terrain Analysis in Realtime Strategy Games. Pottinger (2001?)
Type of Research: Commercial computer game development.
Pottinger, the technical director for Ensemble Studios the creators of Age of
Empires (AOE) and Age of Empires 2 (AOE2) describes the methods used in the
game to analyze and store homogenous terrain types within the game.
AOE, AOE2 and most other Real Time Strategy (RTS) games must support
randomly generated maps. Consequentially, after the random map is generated, but
before the game actually starts, an analysis must quickly be made of the map and
stored for future use.
Figure 3: Influence map around gold Figure 5: Terrain from Figure 4
mines. mapped into subareas.
The technique employed by Pottinger involves mapping ‘influence’ (or what we used
to call ‘spheres of influence’ around key areas. Furthermore, Pottinger divides the
areas into subareas (see Figure 5) above. These areas are determined by a flood fill
algorithm (old but effective) with some modifications to keep from overrunning the
The 3D maps are created using the proprietary BANG! 3D engine which allows for
multiple layers of influence mapping and storage.
It is interesting to note that Pottinger writes, “Pathfinding is one of the slowest
things most RTS games do (e.g. Age of Empires 2 spends roughly 60 to 70% of
simulation time doing pathfinding).” Having played AOE and AOE2, and being
aware of the fairly small size of the maps (usually less than 100 x 100 tiles) I would
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 58 of 116 have thought it would have been possible to pre-calculate paths and store them
leaving the CPU time for other things.
Pottinger concludes with this advice:
• Terrain analysis doesn’t need to be exact. If you try to make it perfect, you’ll either end up
spending a lot of time tweaking heuristics or you’ll have a lot of extra areas that really don’t
buy you anything.
• Abstract the area representation (tile vs. polygonal) away from the users (e.g. the CP AI) as
much as possible. This will allow you to change it to fix bugs or upgrade as needed.
• Build it to support dynamic terrain. Even if you don’t have dynamic terrain in your shipping
game, this helps debugging and testing.
• Write all of the processing so that it is time-sliced. Even if you don’t use it right away,
someone will inevitably code (or ask you to code) a feature that blows out your carefully
constructed time budgets. If you have time slicing built into the system from the start, that’s
not a problem. This also lets you run the terrain analysis during the game much more easily
• Build area specification tools into the scenario editor. Use these to simplify the processing.
There’s no reason why a scenario author can’t put hints for the terrain analysis into the
scenario. This also lets the scenario author tweak how the CP AI thinks about the map.
• Since the random map generator already has to manage areas and connectivity to generate a
decent map, it should just pass that data to the terrain analysis. If the random map also has a
general shape (e.g. AOE2’s Mediterranean map), that information can also simplify the terrain
• Don’t use tiles as a measure of distance. Be trendy and use meters as the unit of measure so
that you can change the tile size without having to rebalance your game data and your AI
• Pattern recognition is a hit or miss proposition. If scenario authors do a little work and know
the basic properties of the game’s random maps are known, real pattern recognition may not
be needed to do effective terrain analysis.
• If real pattern recognition is needed, though, a couple of easy options present themselves.
Since the tile subareas have too much data and convex areas have unknown holes inside the
areas, the tile subareas could be hulled used instead. The data reduction would make
recognition easier and the smaller areas would tend to have fewer holes. Running pattern
matching on the mip-mapped version of the map is also another option.
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 59 of 116 An Exploration into Computer Games and Computer Generated
Forces. Laird (2001)
Type of Research: Academic, funded in part by grant N61339-99-C-0104 from
ONR and NAWCTSD.
Laird write, “When CGFs are used for training (and other purposes), the primary
goal is to replicate the behavior of a human; that is the behavior should be realistic.
Without realistic, human-like behavior, the danger is that human trainees interacting
with the CGFs will have negative training.” In other words, he does not want AI
that is superior to humans or behaves in a non-human manner.
Laird’s group, who created the SOAR language, created the Quakebot, an agent
that plays in Quake II tournaments.
Below is a list of the main tactics the Quakebot uses. These are implemented across the top-level
• Pick up items based on their spawn locations
• Pick up weapons based on their quality
• Abandon collecting items that are missing
• Remember when missing items will respawn
• Use shortest paths to get objects
• Get health and armor if low on them
• Pickup up other good weapons/ammo if close by
• Use circle-strafe (walk sidewise while shooting)
• Move to best distance for current weapon
• Run away if low on health or outmatched by the enemy's weapon
• Go after enemy based on sound of running
• Go where enemy was last seen
• Wait in a corner of a room that can’t be seen by enemy coming into the room
• Go to nearest spawn room after killing enemy
• Go to rooms enemy is often seen in
A flow chart of the Quakebot’s AI is below:
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 60 of 116
attack ander collect-powerups explor
face-item move-to-item stop notice-item-missing
The Quakebot is programmed by rules. An example is:
If predict-enemy is the current operator and
there is an enemy with health 100,
using the blaster, in room #11 and
I am distance 2 from room #3
predict that the enemy will go to room #3
through door #7.
“Inherent to Soar is a learning mechanism, called chunking, that automatically
creates rules that summarize the processing within impasses. Chunking creates
rules that test the aspects of the situation that were relevant during the generation
of a result. The action of the chunk creates the result. Chunking can speed up
problem solving by compiling complex reasoning into a single rule that bypasses the
problem solving in the future. Chunking is not used with the standard Quakebot
because there is little internal reasoning to compile out; however, with anticipation,
there can be a long chain of internal reasoning that takes significant time (a few
seconds) for the Quakebot to generate. In that case, chunking is perfect for learning
rules that eliminate the need for the Quakebot to regenerate the same prediction.
The learned rules are specific to the exact rooms, but that is appropriate because
the predictions are only valid under special circumstances.”
Laird is well-known for SOAR (http://www.eecs.umich.edu/~soar/), his
presentations at the Computer Game Developers Conference (CGDC), his military
research (most notably for the Air Force) and the Quakebot. Laird has consistently
championed the importance of the commercial gaming industry, and its
developments, to the military establishment.
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 61 of 116 On a personal note, I can remember a very abortive attempt by a group from UC-
Davis to use SOAR in conjunction with the CyberWar XXI game that I consulted
on. At the time I thought the problem was with SOAR, itself. However, the
problem may have been with the programmers, instead.
To the best of knowledge SOAR has been used to program only ‘individual’ agents
and not to act as an AI controller of armies, corps or divisions nor to perform any
sort of strategic reasoning.
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 62 of 116 A Modified Naive Bayes Approach for Autonomous Learning in
an Intelligent CGF. Chia; Williams (2003)
Type of Research: Academic (University of Central Florida).
This paper shows the results of a series of experiments with TankSoar (a simple
tank game using the SOAR language) and knowledge acquisition (KA) learning.
Above: a screen capture of TankSoar. The game is played on a 14 x 14 grid and the
rules are extremely simple by wargames standards. A run is between 2 tanks with
different AIs: one is pre-programmed the other employs a learning routine. One is
aggressive the other is non-agressive:
A flowchart for the learning procedure is below:
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 63 of 116
The results of the experiments were not what were expected. The “learning” AI
took longer and longer to “win” and the lessons that they learned were probably
not desirable in the “real world.” See below:
The Current State of Human-Level Artificial Intelligence in Computer Simulations and Wargames. Page 64 of 116 Indeed, the experiment produced only six learned behaviors (see table above). The
four learned behaviors for the “non-aggressive” tanks were to retreat.
Unfortunately, the retreat was simply backing up so the tank was still hit by the
incoming missile. The two learned behaviors for the aggressive tanks were if you
see something on your radar, attack it.
While KA is almost certainly an important tool in creating CGF AI, this experiment,
in my opinion, is almost worthless.
First, one of the goals of the authors’ is, “To implement a truly autonomous agent,
the agent on its own must encode the appropriate attributes which are relevant to
the execution of behaviors. This remains a challenge for the simulation of human
behavior and for furthering an understanding of learning.” However, human
behavior, especially within a military context, is not simply “aggressive” or “non-
General S. L. A. Marshall did a great deal of research into this very subject which
was published in his classic work, “Men Against Fire.” Marshall discovered that less
than 25% of combatants in World War II could be classified as “self starters” (he
believed the actual number was probably closer to 5%). The actions of these “self
starters” determined the actions of the rest of the troops in their immediate
vicinity. If they moved forward, the rest of the platoon moved forward. If they were
killed or were incapacitated, the rest of the platoon ground to a halt. Marshall’s
work greatly influenced my equations used in my wargames to determine when and
why a unit retreated.
Second, the game TankSoar is far too simple to have any military value whatsoever.