bigskymanΤεχνίτη Νοημοσύνη και Ρομποτική

24 Οκτ 2013 (πριν από 3 χρόνια και 1 μήνα)

60 εμφανίσεις

J. A. A. A.

J. B.

M. V.

M. S.


codename: Agent



Object of the Game

Othello is a 2
player game played on an 8x8 square board with each square
holding a disc (white or black). One player chooses white and the other choo
black discs to play

The object of the game is to have the majority of your color discs on the board at
the end of the game

How to start the Game

lack places two black discs and
hite places two white discs as shown in Fig
1. The game alwa
ys begins with this setup.

Figure 1
: Initial setup of the Game

does a player make a move

A move consists of "
" your opponent's disc(s), then "
" the outflanked
disc(s) to your color.


means to place a disc on the board so that your opponent's row (or rows) of
disc(s) is bordered at each end by a disc of your color. (A "row" may be made up of one
or more discs).

Here's one ex

White disc A was already in place on the board. The placement of white disc B outflanks
the row of three black discs.

White flips the outflanked discs

and now the row looks like this:

Othello Rules

Black always move


If on your turn you cannot outflank and flip at least one opposing disc, you
r turn is
forfeited and your opponent moves again. However, if a move is available to you, you
may not forfeit your turn.

A disc may outflank any number of discs in one or more rows in any number of
directions at the same time

horizontally, verticall
y or diagonally. (A row is defined as
one or more discs in a continuous straight line). See Figures 2 and 3.

Figure 2

Figure 3

You may not skip over your own color disc to outflank an opposing disc. (See Figure

Figure 4

Discs may only be outflanked as a direct result of a move and must fall in the direct
line of the disc placed down. (See Figures 5 and 6).

Figure 5

Figure 6

l discs outflanked in any one move must be flipped, even if it is to the player's
advantage not to flip them at all.

When it is no longer possible for either player to move, the game is over. Discs are
counted and the player with the majority of his or
her color discs on the board is the

: It is possible for a game to end before all 64 squares are filled.

Minimax Algorithm

The minimax algorithm
is used to give

values to nodes

a tree that represent

a game.
The terminal nodes, leaf node
s, are given utility values according to whether they are
good for one player or the other. If the values are high positives, then those values are
good for one player.
, if the values are high negatives, then those values are
good for the other p

Minimax performs recursion on the tree to give values to the nodes that are not leaves.
The value is selected depending on whether the current leave is a move that could be
made by one player or the other. If it corresponds to the first player, the
n the v
selected will be the biggest

value of the children of that node and vice versa.

This algorithm is optimal for games that have a small finite amount of states. For games
like Chess or Go the algorithm is impossible to compute.

beta prun

The alpha
beta pruning algorithm is used to improve the minimax algor
ithm by reducing
(pruning) the
number of comparisons and retrievals the minimax algorithm has to do. It

by analyzing the values it already has and figuring out if it needs to

checking othe
r values from the current leave

it is

The following example is
taken from the book and illustrates the algorithm:


= max(min(3,12,8),min(2,x,y),min(14,5,2))

= max(3,min(2,x,y),2)

= max(3,z,2)
where z<= 2

= 3

The example shows that the values of x and y are not needed
the algorithm

select the lowest value between 2, x, and y, only to then select the highest value from 3,
(2, x, or y),
2, which is 3. So, finding out what x a
nd y are is not necessary

for the
algorithm to work

Neural Networks

Neural Networks basic conception stems from trying to model brain synapses and
neurons. The idea is that when you learn, certain paths are reinforced. These reinforced
paths contain
either new knowledge or make it easier to recall old knowledge.
Unfortunately the abstract models currently in existence are pale shadows compared to
the makeup of the ones we as humans use. This is not to say these models are not useful;
actually the op
posite is quite true. They are used in many different systems to develop
new answers to difficult problems.

Make Up

Basically Neural Networks use a series of nodes that are distributed in different layers
(mimicking a tree like design), to simulate dec
ision making. A neural network is
required to have an input layer and output layer at the very least. There are sometimes
layers called hidden layers that are simply just another layer between the input and output
layers. Each of the nodes in the system

will give a certain output. This is dependent on
the input given to the node and the weight you give certain paths out of the node. This,
of course, makes the entire structure configurable.

Why Use

We decided
to use them for

several reasons. F
irst off the network can output different
information depending on the input given to it. This is very applicable with Othello
because there are a lot of different board configurations possible in any given game.
Secondly, the entire system is configurab
le due to the weights on any given node. This
makes the entire system very flexible. It also allows the system to have certain
“learning” qualities. For example, if our Neural Network is given certain input it will
generate a move. If this move is a ve
ry poor one then we can adjust he weights to give
this move lower priority.

Genetic Algorithms


Genetic algorithms

are a part of
evolutionary computing
, which is a rapidly growing
area of artificial intelligence.

Search Space

If we are sol
ving a problem, we are usually looking for some solution which will be the
best among others. The space of all feasible solutions (the set of solutions among which
the desired solution resides) is called
search space

(also state space). Each point in the
earch space represents one possible solution. Each possible solution can be "marked" by
its value (or fitness) for the problem. With GA we look for the best solution among a
number of possible solutions

represented by one point in the search space.

ing for a solution is then equal to looking for some extreme value (minimum or
maximum) in the search space. At times the search space may be well defined, but
usually we know only a few points in the search space. In the process of using GA, the
process o
f finding solutions generates other points (possible solutions) as evolution

Basic Description of Genetic Algorithms

Genetic algorithms are inspired by Darwin's theory of evolution. Solution to a problem
solved by genetic algorithms uses an ev
olutionary process (it is evolved).

Algorithm begins with a
set of solutions

(represented by
) called
. Solutions from one population are taken and used to form a new population.
This is motivated by a hope, that the new population w
ill be better than the old one.
Solutions which are then selected to form new solutions (offspring) are selected
according to their fitness

the more suitable they are the more chances they have to

This process of generating successive genera
tions of solutions is repeated until some
condition (for example number of populations or improvement of the best solution) is

Encoding of a Chromosome

A chromosome should in some way contain information about solution that it represents.
e most used way of encoding is a binary string. A chromosome then could look like

Chromosome 1


Chromosome 2


Each chromosome is represented by a binary string. Each bit in the string can represent
some characteris
tics of the solution. There are many other ways of encoding. The
encoding depends mainly on the solved problem. For example, one can encode directly
integer or real numbers (which is the case for generating weights of a Neural Network);
sometimes it is use
ful to encode some permutations and so on.


After we have decided what encoding we will use, we can proceed to crossover
operation. Crossover operates on selected genes from parent chromosomes and creates
new offspring. The simplest way how to

do that is to choose randomly some crossover
point and copy everything before this point from the first parent and then copy everything
after the crossover point from the other parent.

Crossover can be illustrated as follows: (| is the crossover point):

Chromosome 1

11011 | 00100110110

Chromosome 2

11011 | 11000011110

Offspring 1



Offspring 2



There are other ways how to make crossover, for example we can choose more crossover
points. Crossover can be quite com
plicated and depends mainly on the encoding of
chromosomes. Specific crossover made for a specific problem can improve performance
of the genetic algorithm.


After a crossover is performed, mutation takes place. Mutation is intended to prevent
falling of all solutions in the population into a local optimum of the solved problem.
Mutation operation randomly changes the offspring resulted from crossover. In case of
binary encoding we can switch a few randomly chosen bits from 1 to 0 or from 0 to 1
Mutation can be then illustrated as follows:

Original offspring 1


Original offspring 2


Mutated offspring 1


Mutated offspring 2


The technique of mutation (as well as crossover) depe
nds mainly on the encoding of
chromosomes. For example when we are encoding permutations, mutation could be
performed as an exchange of two genes.

Parameters of GA

There are two basic parameters of GA

crossover probability and mutation probability.

Crossover probability
: How often crossover will be performed. If there is no crossover,
offspring are exact copies of parents. If there is crossover, offspring are made from parts
of both parent's chromosome. If crossover probability is 100%, then all offs
pring are
made by crossover. If it is 0%, whole new generation is made from exact copies of
chromosomes from old population (but this does not mean that the new generation is the
same!). Crossover is made in hope that new chromosomes will contain good part
s of old
chromosomes and therefore the new chromosomes will be better. However, it is good to
leave some part of old population survive to the next generation.

Mutation probability
: How often parts of chromosome will be mutated. If there is no
offspring are generated immediately after crossover (or directly copied)
without any change. If mutation is performed, one or more parts of a chromosome are
changed. If mutation probability is 100%, whole chromosome is changed, if it is 0%,
nothing is chan
ged. Mutation generally prevents the GA from falling into local extremes.
Mutation should not occur very often, because then GA will in fact change to random

Other Parameters

There are also some other parameters of GA. One another particularly
parameter is population size.

Population size
: How many chromosomes are in population (in one generation). If there
are too few chromosomes, GA have few possibilities to perform crossover and only a
small part of search space is explored. On th
e other hand, if there are too many
chromosomes, GA slows down. Research shows that after some limit (which depends
mainly on encoding and the problem) it is not useful to use very large populations
because it does not solve the problem faster than moderat
e sized populations.


of the system



Research specific components of the agent


Continue researching


Requirements and decide on design


Tutorial completion


Low level design


Start coding


Keep coding


Don’t stop


You are almost there


totype ready


Keep working on prototype


Celebrate victory

over other team


The Othello rules were taken from

The genetic algorithms explanation was taken from


oth the rules and the genetic algorithm information were slightly modified to preserve
the overall structure of the tutorial. No corrections were made in the grammar of the
information to preserve copyright.