# Tutorial

Τεχνίτη Νοημοσύνη και Ρομποτική

24 Οκτ 2013 (πριν από 3 χρόνια και 1 μήνα)

60 εμφανίσεις

J. A. A. A.

J. B.

M. V.

M. S.

Project

codename: Agent
D
ouble
O

111

Othello
Tutorial

Object of the Game

Othello is a 2
-
player game played on an 8x8 square board with each square
holding a disc (white or black). One player chooses white and the other choo
ses
black discs to play

The object of the game is to have the majority of your color discs on the board at
the end of the game

How to start the Game

Initially,
B
lack places two black discs and
W
hite places two white discs as shown in Fig
1. The game alwa
ys begins with this setup.

Figure 1
: Initial setup of the Game

How
does a player make a move

A move consists of "
outflanking
" your opponent's disc(s), then "
flipping
" the outflanked

To
outflank

means to place a disc on the board so that your opponent's row (or rows) of
disc(s) is bordered at each end by a disc of your color. (A "row" may be made up of one
or more discs).

Here's one ex
ample:

White disc A was already in place on the board. The placement of white disc B outflanks
the row of three black discs.

White flips the outflanked discs

and now the row looks like this:

Othello Rules

1.
Black always move
s

first.

2.
If on your turn you cannot outflank and flip at least one opposing disc, you
r turn is
forfeited and your opponent moves again. However, if a move is available to you, you

3.
A disc may outflank any number of discs in one or more rows in any number of
directions at the same time
-

horizontally, verticall
y or diagonally. (A row is defined as
one or more discs in a continuous straight line). See Figures 2 and 3.

Figure 2

Figure 3

4.
You may not skip over your own color disc to outflank an opposing disc. (See Figure
4).

Figure 4

5.
Discs may only be outflanked as a direct result of a move and must fall in the direct
line of the disc placed down. (See Figures 5 and 6).

Figure 5

Figure 6

6.
Al
l discs outflanked in any one move must be flipped, even if it is to the player's
advantage not to flip them at all.

7.
When it is no longer possible for either player to move, the game is over. Discs are
counted and the player with the majority of his or
her color discs on the board is the
winner.

NOTE
: It is possible for a game to end before all 64 squares are filled.

Minimax Algorithm

The minimax algorithm
is used to give

values to nodes
of

a tree that represent

a game.
The terminal nodes, leaf node
s, are given utility values according to whether they are
good for one player or the other. If the values are high positives, then those values are
good for one player.
Otherwise
, if the values are high negatives, then those values are
good for the other p
layer.

Minimax performs recursion on the tree to give values to the nodes that are not leaves.
The value is selected depending on whether the current leave is a move that could be
made by one player or the other. If it corresponds to the first player, the
n the v
alue
selected will be the biggest

value of the children of that node and vice versa.

This algorithm is optimal for games that have a small finite amount of states. For games
like Chess or Go the algorithm is impossible to compute.

Alpha
-
beta prun
ing

The alpha
-
beta pruning algorithm is used to improve the minimax algor
ithm by reducing
(pruning) the
number of comparisons and retrievals the minimax algorithm has to do. It
does
so

by analyzing the values it already has and figuring out if it needs to

continue
checking othe
r values from the current leave

it is
evaluating
.

The following example is
taken from the book and illustrates the algorithm:

MINIMAX
-
VALUE(root)

= max(min(3,12,8),min(2,x,y),min(14,5,2))

= max(3,min(2,x,y),2)

= max(3,z,2)
where z<= 2

= 3

The example shows that the values of x and y are not needed
because
the algorithm

will
select the lowest value between 2, x, and y, only to then select the highest value from 3,
(2, x, or y),
and
2, which is 3. So, finding out what x a
nd y are is not necessary

for the
algorithm to work
.

Neural Networks

Neural Networks basic conception stems from trying to model brain synapses and
neurons. The idea is that when you learn, certain paths are reinforced. These reinforced
paths contain
either new knowledge or make it easier to recall old knowledge.
Unfortunately the abstract models currently in existence are pale shadows compared to
the makeup of the ones we as humans use. This is not to say these models are not useful;
actually the op
posite is quite true. They are used in many different systems to develop

Make Up

Basically Neural Networks use a series of nodes that are distributed in different layers
(mimicking a tree like design), to simulate dec
ision making. A neural network is
required to have an input layer and output layer at the very least. There are sometimes
layers called hidden layers that are simply just another layer between the input and output
layers. Each of the nodes in the system

will give a certain output. This is dependent on
the input given to the node and the weight you give certain paths out of the node. This,
of course, makes the entire structure configurable.

Why Use
Them
?

We decided
to use them for

several reasons. F
irst off the network can output different
information depending on the input given to it. This is very applicable with Othello
because there are a lot of different board configurations possible in any given game.
Secondly, the entire system is configurab
le due to the weights on any given node. This
makes the entire system very flexible. It also allows the system to have certain
“learning” qualities. For example, if our Neural Network is given certain input it will
generate a move. If this move is a ve
ry poor one then we can adjust he weights to give
this move lower priority.

Genetic Algorithms

Introduction

Genetic algorithms

are a part of
evolutionary computing
, which is a rapidly growing
area of artificial intelligence.

Search Space

If we are sol
ving a problem, we are usually looking for some solution which will be the
best among others. The space of all feasible solutions (the set of solutions among which
the desired solution resides) is called
search space

(also state space). Each point in the
s
earch space represents one possible solution. Each possible solution can be "marked" by
its value (or fitness) for the problem. With GA we look for the best solution among a
number of possible solutions
-

represented by one point in the search space.

Look
ing for a solution is then equal to looking for some extreme value (minimum or
maximum) in the search space. At times the search space may be well defined, but
usually we know only a few points in the search space. In the process of using GA, the
process o
f finding solutions generates other points (possible solutions) as evolution
proceeds.

Basic Description of Genetic Algorithms

Genetic algorithms are inspired by Darwin's theory of evolution. Solution to a problem
solved by genetic algorithms uses an ev
olutionary process (it is evolved).

Algorithm begins with a
set of solutions

(represented by
chromosomes
) called
population
. Solutions from one population are taken and used to form a new population.
This is motivated by a hope, that the new population w
ill be better than the old one.
Solutions which are then selected to form new solutions (offspring) are selected
according to their fitness
-

the more suitable they are the more chances they have to
reproduce.

This process of generating successive genera
tions of solutions is repeated until some
condition (for example number of populations or improvement of the best solution) is
satisfied.

Encoding of a Chromosome

A chromosome should in some way contain information about solution that it represents.
Th
e most used way of encoding is a binary string. A chromosome then could look like
this:

Chromosome 1

1101100100110110

Chromosome 2

1101111000011110

Each chromosome is represented by a binary string. Each bit in the string can represent
some characteris
tics of the solution. There are many other ways of encoding. The
encoding depends mainly on the solved problem. For example, one can encode directly
integer or real numbers (which is the case for generating weights of a Neural Network);
sometimes it is use
ful to encode some permutations and so on.

Crossover

After we have decided what encoding we will use, we can proceed to crossover
operation. Crossover operates on selected genes from parent chromosomes and creates
new offspring. The simplest way how to

do that is to choose randomly some crossover
point and copy everything before this point from the first parent and then copy everything
after the crossover point from the other parent.

Crossover can be illustrated as follows: (| is the crossover point):

Chromosome 1

11011 | 00100110110

Chromosome 2

11011 | 11000011110

Offspring 1

11011

|
11000011110

Offspring 2

11011
|

00100110110

There are other ways how to make crossover, for example we can choose more crossover
points. Crossover can be quite com
plicated and depends mainly on the encoding of
chromosomes. Specific crossover made for a specific problem can improve performance
of the genetic algorithm.

Mutation

After a crossover is performed, mutation takes place. Mutation is intended to prevent
falling of all solutions in the population into a local optimum of the solved problem.
Mutation operation randomly changes the offspring resulted from crossover. In case of
binary encoding we can switch a few randomly chosen bits from 1 to 0 or from 0 to 1
.
Mutation can be then illustrated as follows:

Original offspring 1

1101111000011110

Original offspring 2

1101100100110110

Mutated offspring 1

1100111000011110

Mutated offspring 2

1101101100110110

The technique of mutation (as well as crossover) depe
nds mainly on the encoding of
chromosomes. For example when we are encoding permutations, mutation could be
performed as an exchange of two genes.

Parameters of GA

There are two basic parameters of GA
-

crossover probability and mutation probability.

Crossover probability
: How often crossover will be performed. If there is no crossover,
offspring are exact copies of parents. If there is crossover, offspring are made from parts
of both parent's chromosome. If crossover probability is 100%, then all offs
pring are
made by crossover. If it is 0%, whole new generation is made from exact copies of
chromosomes from old population (but this does not mean that the new generation is the
same!). Crossover is made in hope that new chromosomes will contain good part
s of old
chromosomes and therefore the new chromosomes will be better. However, it is good to
leave some part of old population survive to the next generation.

Mutation probability
: How often parts of chromosome will be mutated. If there is no
mutation,
offspring are generated immediately after crossover (or directly copied)
without any change. If mutation is performed, one or more parts of a chromosome are
changed. If mutation probability is 100%, whole chromosome is changed, if it is 0%,
nothing is chan
ged. Mutation generally prevents the GA from falling into local extremes.
Mutation should not occur very often, because then GA will in fact change to random
search.

Other Parameters

There are also some other parameters of GA. One another particularly
important
parameter is population size.

Population size
: How many chromosomes are in population (in one generation). If there
are too few chromosomes, GA have few possibilities to perform crossover and only a
small part of search space is explored. On th
e other hand, if there are too many
chromosomes, GA slows down. Research shows that after some limit (which depends
mainly on encoding and the problem) it is not useful to use very large populations
because it does not solve the problem faster than moderat
e sized populations.

Diagram

of the system

Timeline

1)

Research specific components of the agent

2)

Continue researching

3)

Requirements and decide on design

4)

Tutorial completion

5)

Low level design

6)

Start coding

7)

Keep coding

8)

Don’t stop

9)

You are almost there

10)

Pro

11)

Keep working on prototype

12)

Celebrate victory

over other team

References

The Othello rules were taken from
htt
p://www.rainfall.com/othello/rules/othellorules.asp

The genetic algorithms explanation was taken from

http://cs.felk.cvut.cz/~xobitko/ga/

B
oth the rules and the genetic algorithm information were slightly modified to preserve
the overall structure of the tutorial. No corrections were made in the grammar of the