J. A. A. A.
Object of the Game
Othello is a 2
player game played on an 8x8 square board with each square
holding a disc (white or black). One player chooses white and the other choo
black discs to play
The object of the game is to have the majority of your color discs on the board at
the end of the game
How to start the Game
lack places two black discs and
hite places two white discs as shown in Fig
1. The game alwa
ys begins with this setup.
: Initial setup of the Game
does a player make a move
A move consists of "
" your opponent's disc(s), then "
" the outflanked
disc(s) to your color.
means to place a disc on the board so that your opponent's row (or rows) of
disc(s) is bordered at each end by a disc of your color. (A "row" may be made up of one
or more discs).
Here's one ex
White disc A was already in place on the board. The placement of white disc B outflanks
the row of three black discs.
White flips the outflanked discs
and now the row looks like this:
Black always move
If on your turn you cannot outflank and flip at least one opposing disc, you
r turn is
forfeited and your opponent moves again. However, if a move is available to you, you
may not forfeit your turn.
A disc may outflank any number of discs in one or more rows in any number of
directions at the same time
y or diagonally. (A row is defined as
one or more discs in a continuous straight line). See Figures 2 and 3.
You may not skip over your own color disc to outflank an opposing disc. (See Figure
Discs may only be outflanked as a direct result of a move and must fall in the direct
line of the disc placed down. (See Figures 5 and 6).
l discs outflanked in any one move must be flipped, even if it is to the player's
advantage not to flip them at all.
When it is no longer possible for either player to move, the game is over. Discs are
counted and the player with the majority of his or
her color discs on the board is the
: It is possible for a game to end before all 64 squares are filled.
The minimax algorithm
is used to give
values to nodes
a tree that represent
The terminal nodes, leaf node
s, are given utility values according to whether they are
good for one player or the other. If the values are high positives, then those values are
good for one player.
, if the values are high negatives, then those values are
good for the other p
Minimax performs recursion on the tree to give values to the nodes that are not leaves.
The value is selected depending on whether the current leave is a move that could be
made by one player or the other. If it corresponds to the first player, the
n the v
selected will be the biggest
value of the children of that node and vice versa.
This algorithm is optimal for games that have a small finite amount of states. For games
like Chess or Go the algorithm is impossible to compute.
beta pruning algorithm is used to improve the minimax algor
ithm by reducing
number of comparisons and retrievals the minimax algorithm has to do. It
by analyzing the values it already has and figuring out if it needs to
r values from the current leave
The following example is
taken from the book and illustrates the algorithm:
where z<= 2
The example shows that the values of x and y are not needed
select the lowest value between 2, x, and y, only to then select the highest value from 3,
(2, x, or y),
2, which is 3. So, finding out what x a
nd y are is not necessary
algorithm to work
Neural Networks basic conception stems from trying to model brain synapses and
neurons. The idea is that when you learn, certain paths are reinforced. These reinforced
either new knowledge or make it easier to recall old knowledge.
Unfortunately the abstract models currently in existence are pale shadows compared to
the makeup of the ones we as humans use. This is not to say these models are not useful;
actually the op
posite is quite true. They are used in many different systems to develop
new answers to difficult problems.
Basically Neural Networks use a series of nodes that are distributed in different layers
(mimicking a tree like design), to simulate dec
ision making. A neural network is
required to have an input layer and output layer at the very least. There are sometimes
layers called hidden layers that are simply just another layer between the input and output
layers. Each of the nodes in the system
will give a certain output. This is dependent on
the input given to the node and the weight you give certain paths out of the node. This,
of course, makes the entire structure configurable.
to use them for
several reasons. F
irst off the network can output different
information depending on the input given to it. This is very applicable with Othello
because there are a lot of different board configurations possible in any given game.
Secondly, the entire system is configurab
le due to the weights on any given node. This
makes the entire system very flexible. It also allows the system to have certain
“learning” qualities. For example, if our Neural Network is given certain input it will
generate a move. If this move is a ve
ry poor one then we can adjust he weights to give
this move lower priority.
are a part of
, which is a rapidly growing
area of artificial intelligence.
If we are sol
ving a problem, we are usually looking for some solution which will be the
best among others. The space of all feasible solutions (the set of solutions among which
the desired solution resides) is called
(also state space). Each point in the
earch space represents one possible solution. Each possible solution can be "marked" by
its value (or fitness) for the problem. With GA we look for the best solution among a
number of possible solutions
represented by one point in the search space.
ing for a solution is then equal to looking for some extreme value (minimum or
maximum) in the search space. At times the search space may be well defined, but
usually we know only a few points in the search space. In the process of using GA, the
f finding solutions generates other points (possible solutions) as evolution
Basic Description of Genetic Algorithms
Genetic algorithms are inspired by Darwin's theory of evolution. Solution to a problem
solved by genetic algorithms uses an ev
olutionary process (it is evolved).
Algorithm begins with a
set of solutions
. Solutions from one population are taken and used to form a new population.
This is motivated by a hope, that the new population w
ill be better than the old one.
Solutions which are then selected to form new solutions (offspring) are selected
according to their fitness
the more suitable they are the more chances they have to
This process of generating successive genera
tions of solutions is repeated until some
condition (for example number of populations or improvement of the best solution) is
Encoding of a Chromosome
A chromosome should in some way contain information about solution that it represents.
e most used way of encoding is a binary string. A chromosome then could look like
Each chromosome is represented by a binary string. Each bit in the string can represent
tics of the solution. There are many other ways of encoding. The
encoding depends mainly on the solved problem. For example, one can encode directly
integer or real numbers (which is the case for generating weights of a Neural Network);
sometimes it is use
ful to encode some permutations and so on.
After we have decided what encoding we will use, we can proceed to crossover
operation. Crossover operates on selected genes from parent chromosomes and creates
new offspring. The simplest way how to
do that is to choose randomly some crossover
point and copy everything before this point from the first parent and then copy everything
after the crossover point from the other parent.
Crossover can be illustrated as follows: (| is the crossover point):
11011 | 00100110110
11011 | 11000011110
There are other ways how to make crossover, for example we can choose more crossover
points. Crossover can be quite com
plicated and depends mainly on the encoding of
chromosomes. Specific crossover made for a specific problem can improve performance
of the genetic algorithm.
After a crossover is performed, mutation takes place. Mutation is intended to prevent
falling of all solutions in the population into a local optimum of the solved problem.
Mutation operation randomly changes the offspring resulted from crossover. In case of
binary encoding we can switch a few randomly chosen bits from 1 to 0 or from 0 to 1
Mutation can be then illustrated as follows:
Original offspring 1
Original offspring 2
Mutated offspring 1
Mutated offspring 2
The technique of mutation (as well as crossover) depe
nds mainly on the encoding of
chromosomes. For example when we are encoding permutations, mutation could be
performed as an exchange of two genes.
Parameters of GA
There are two basic parameters of GA
crossover probability and mutation probability.
: How often crossover will be performed. If there is no crossover,
offspring are exact copies of parents. If there is crossover, offspring are made from parts
of both parent's chromosome. If crossover probability is 100%, then all offs
made by crossover. If it is 0%, whole new generation is made from exact copies of
chromosomes from old population (but this does not mean that the new generation is the
same!). Crossover is made in hope that new chromosomes will contain good part
s of old
chromosomes and therefore the new chromosomes will be better. However, it is good to
leave some part of old population survive to the next generation.
: How often parts of chromosome will be mutated. If there is no
offspring are generated immediately after crossover (or directly copied)
without any change. If mutation is performed, one or more parts of a chromosome are
changed. If mutation probability is 100%, whole chromosome is changed, if it is 0%,
nothing is chan
ged. Mutation generally prevents the GA from falling into local extremes.
Mutation should not occur very often, because then GA will in fact change to random
There are also some other parameters of GA. One another particularly
parameter is population size.
: How many chromosomes are in population (in one generation). If there
are too few chromosomes, GA have few possibilities to perform crossover and only a
small part of search space is explored. On th
e other hand, if there are too many
chromosomes, GA slows down. Research shows that after some limit (which depends
mainly on encoding and the problem) it is not useful to use very large populations
because it does not solve the problem faster than moderat
e sized populations.
of the system
Research specific components of the agent
Requirements and decide on design
Low level design
You are almost there
Keep working on prototype
over other team
The Othello rules were taken from
The genetic algorithms explanation was taken from
oth the rules and the genetic algorithm information were slightly modified to preserve
the overall structure of the tutorial. No corrections were made in the grammar of the
information to preserve copyright.