GP End-Chess

ghostslimΤεχνίτη Νοημοσύνη και Ρομποτική

23 Φεβ 2014 (πριν από 3 χρόνια και 4 μήνες)

65 εμφανίσεις

1

GP End
-
Chess


Evolution of Chess Endgame Players


Ami Hauptman & Moshe Sipper

2

Outline


Introduction


The Game of Chess


a solved problem?


Important differences between human and artificial chess
players


Chess Endgames
-

features & building blocks


GP problem definition


Experiment and Results


Future work

3

The game of Chess


First developed in India and Persia


Considered THE complex game of strategy and
inventiveness


Enormous search space


Roughly
50
possible moves at mid
-
game


A typical game consists of a few dozen moves


Estimated at
10
43

in
40
-
move game (Shannon,
1950
)




Elephants don

t play Chess(?)

4

The game of Chess


AI history


First chess AI at
1958


novice level


Machine strength increasing linearly


1997


defeat of former world champion, Garry
Kasparov, by IBM’s deep blue


Last years


performance still increasing


Mainly Hardware


Also Software



The End ?





5

The game of Chess


…NO!


Deep(er) blue use extreme brute
-
force,
traversing several millions of boards ps


Very little generalization


Virtually no human resemblance


Deemed theoretically uninteresting


Chomsky: As interesting as a weight lifting competition
between machine and man


Low A to I ratio; Low return




6


The game of Chess



Basic concepts


8
x
8
board


Each player starts with
16
pieces, of
6
different types, and may only move
1
piece
per turn


A piece can only move into an empty square or
into one containing an opponent’s piece (
a
capture
)


Win by capturing the opponent’s king




7

The game of Chess


pieces


Pawn: may only move forward (or capture diagonally)


Bishop: diagonals


Knight: L shaped moves. The only “unblockable” piece


Rook:

Ranks & files


Queen:

Bishop & Rook combined


King:

1
square in any direction. May not move into
attacked square

Values :

1 3 3
/
3.5 5 9



8

The game of Chess


example



White has over
30
possible moves




If black

s turn


can capture pawn
at c
3
and check
(also fork)



9

The game of Chess



Check and Checkmate


“Checking” is attacking
opponent’s king.
Opponent
must

respond



“Mating” (Checkmate)


is when the opponent
can’t avoid losing the
king


and thus
forfeiting the game

10

Man vs Machine

11

Human & Artificial Players



AI search


AI uses
search

to
assign a score to a
board


Traverse the move
tree from leaves
-

up


Select the best child
using scores found


Only
partial

tree





Computer is O (Max) opponent is X

12

Human & Artificial Players



The Machine



13

Human & Artificial Players



The Machine


Millions of boards (nodes) per second


Little time for each board


less knowledge


Smart search algorithms




pruning


Alpha
-
beta variants (negascout etc.)


Still use heuristics at end


can’t see all tree


Most research revolves around search


Human resemblance minimal


humans use
little search



14

Human & Artificial Players


Humans


Humans use problem solving cognition


Deeply knowledge based




Extensive “theory” exists


Numerous books and institutions


Massive use of pattern recognition


Also use search but


Less deep


Only develop “good” positions


More

efficient


less nodes for “same” result


Reminiscent of greedy search


Not only in chess

15

Human & Artificial Players


Grand Masters
-

Findings


Play against several opponents at the same
level they play against a single opponent


GMs and novices: same level of performance
when memorizing a random board; differ when
memorizing real game positions


GM eye movements show they only scan
“correct” parts of board


Strong Amateurs use the same meta
-
search as
GMs
-

equally deep, same nodes, same speed;
Differ in knowledge of domain (De Groot)

16


Endgames
-

example


White’s turn: mate in
5
,
with Qe
6
+


Features include:


#moves for black king
minimal


Attacking, un
-
attacked


Checking


Officers same line
\
row


Black’s turn: draw with:
Rc
1
+, then Qg
5


fork
& exchange

17


Endgames (
2
)
-

features



Few pieces remain (typically: king,
0
-
3
officers and sometimes pawns)


Fewer options, but more moves for each piece


Trees still extremely large



18

Endgames
-

Building Blocks


Main goals


Reduce search by “smart” features of the board


Use more game knowledge as humans do


Allow more complex features to be built by
supplying basic ones (terminals) and building
methods (functions)


Schemata evolution


19

Features Example
-

Fork


My piece is:


Attacking
2
or more pieces


Protected or not attacked


Opponent pieces:


Unprotected


OR protected but of greater
value


Example: black must
exchange Q for R because
of fork

20

Fork: Traditional AI search


Only
3
legal moves for
black


Find that one of white’s
next moves (out of
23
possible) captures black
queen


Check all following moves
for more piece exchanges


Sometimes, still check
other moves (non
capturing)


At end of search


compare remaining pieces


No fork “concept”

21

Features Example


Fork

Feature Search (GP)



One of the features is
isMyFork

function


Checks
all previously defined
conditions


Also, use some smaller
building blocks:


Is Opponent piece Attacked?


Is attacking piece protected?


Is opponent in check?


Value of attacked piece


22

GP Problem Definition


Terminals & Functions


Numerous “chess terminals” and ERCs


Non
-
chess funtions


Fitness


Tournament


Run parameters


Termination


We will see each element in the following
experiments

23

Endgame experiments conducted


KRKR



each player has
1
king and
1
rook


KRRKR



King with
2
Rook against King and Rook


KRRKRR


KQKQ



Kings and Queens


KQRKQR



Combined

24

Basic program architecture


Generate all possible moves (depth=
1
)


Evaluate each board with GP individual


Select board with best score (or stochastically
decide between equal)


Perform best move


Repeat process with GP opponent until game ends
(or until only kings left)

25

KRKR Endgame


Each player has
1
King,
1
Rook


“Toy” problem for chess endgames


Theoretical draw (experts never lose this)


Some interesting positions exist



26

KRKR Endgame
-


what needs to be learned (
1
)


Avoid losing rook


Don’t move to
attacked, unprotected
squares



Vice versa
-

capture
opponent’s rook if
able


Black to move


white loses Rook

27

KRKR Endgame
-


what need to be learned (
2
)


Avoid getting king
stuck in edges



Again, take
advantage if
opponent does this

Black to move


mate in
1

28

KRKR Endgames
-

Terminals


Used in first runs:


Is
My

Rook Attacked, Is
Opp

Rook
attacked


Is {
My, Opp
} Rook Protected (two as
above)


Is {
My, Opp
} Rook In Play


Num Moves {
My, Opp
} king


{
My, Opp
}
-
King’s proximity to edges


Is Mate


ERCs:
±

{
0.25
,
0.5
,
1
} *
MAX


MAX =
1000
(empirically)


29


KRKR Endgames
-

Functions


Boolean


OR
2
, OR
3
, OR
4


AND
2
, AND
3
, AND
4


NOT


Arithmetic
-

+,
-
, *


Combined
-

<, =, >, IF


STGP


For now, no “chess” functions, only
terminals

30


KRKR Endgames
-

Fitness


Competitive, Random
-
2
-
ways


Each individual plays against
k

randomly selected
opponents


Each game counts for both players


For each encounter


Several games (typically
4
) are played


Short games
-

~
5
-
8
moves per player


Each game starts at a random legal position


Safe start
-

no piece is attacked at the beginning

31


KRKR Endgames


Fitness

(
2
)


Scoring method:


Victory:
1
-
2
points


Piece count advantage (theoretical win)


¾ point


Draw: ½ point


After advantage


0
points


Loss:
0
points


32


KRKR Endgames


Parameters


Population size
-

80


#Generations
-

150
..
250


Operators:


Reproduction
0.35


Crossover
0.5


Mutation
0.15
(including ERC mutation)


Termination


~
10
-
25
hours


33


KRKR Endgames


Results


Every
10
generations, best individual played
against:


Best of generation
0


An opponent performing random moves


Longer games: ~
10
-
12
moves per player


50
-
150
games


Games were doubled


each player staring
from both positions



34


KRKR Endgames


Results


Bad results



no distinct improvement


Several reasons:


Arithmetic operations problematic


we get large
numbers


Mate not distinct enough (traditionally terminates
the search)


Boolean functions not clear enough


Slow Runs due to large trees with repeating
functions

35


KRKR Endgames

Improvements


Boolean functions


Divided to good and bad


Example:
Is
-
My
-
King
-
In
-
Check

changed to
Is
-
My
-
King
-
Not
-
In
-
Check


Mate changed to
1000
*Mate


Added
Not
-
My
-
Rook
-
Attacked
-
Unprotected

and
Opp
-
Rook
-
Attacked
-
Unprotected

36

KRKR Endgames


Results
-

Improvements


Also consulted Chess
Experts


added more:


Is
-
Opp
-
King
-
Behind
-
Rook


Split to


Opp
-
King
-
Prox
-
Rook


Opp
-
King
-
Behind
-
Rook


Is
-
Stalemate

(only kings
left)

Black moves and White loses Rook

37


KRKR Endgames


Results
-

Improvements


Arithmetic functions canceled


Although Still using floats for terminals


Also divide to good and bad: Num
Not
MovesOppKing


Theoretically justified


more “logical” search in
literature


Empirically
-

need more logical rules, and
not

: (
> (+ (#moves
-
k #moves
-
opp
-
k)
5.5
))


Memoization


saves more than ½ the time








38


KRKR Endgames



Final Results


Improvement


Above
75
% of games against random end in
advantage or mate


Still, too few mates, even when score for win is
increased


difficult to learn move sequence


Same against best of generation
0
(almost random)


The main thing that was learned was avoiding
getting the rook captured










39

KRRKR Endgames


Example (right)



Very good for
white


Black king exposed


2
rooks close


Next move


captures rook


(mate in
5
)


40

KRRKR Endgames
-

goals


One player has
2
rooks, the other


1


Not theoretically drawn


We want one generalized individual for all
endgames and positions (Not one for each
endgame):


Each player needs to play both advantage, draw
(KRKR) and disadvantage


Terminals need to be more general



41

KRRKR Endgames
-

changes


Terminals
-

changed and added to cope with
changing state


Material
-
Count
(recall each rook =
5
)


Num
-
My
-
Pieces
-
Not
-
Attacked
, since now there
are more than
1


Is
-
My
-
King
-
Protecting
-
Piece

and
My
-
Officers
-
Same
-
Line

to allow more complex considerations


Functions


If
-
Adv
-
Then
-
(
left child
)
-
Else
-
(
right child
)


Eventually divided to
3
trees

42

KRRKR Endgames
-

changes


Also added


comparing differences to parent
node


Boolean
Is
-
Material
-
Increase
, which compares to
the parent node (board)


Material decrease is not needed since considering
only my move


Not
-
My
-
King
-
Moves
-
Decrease

to further use
number of moves for king





43

KRRKR Endgames



Opponents


Random forsaken; Best
-
of
-
0
still used but less


Added new opponent


MASTER


a program we wrote based on consultation with
experts, highest being
International

Master

Boris
Gutkin, ELO
2400
(only about
3000
of those…)


Used ~
50
general positions and rules derived from
them, together with scores for each


Defined a strategy (“
Expert
”) accordingly


Tested evolved programs against it


Human competitive?

44

KRRKR Endgames



Fitness


Test were conducted by assigning each player
both roles for each position


Fitness was refined


score effected by:


Starting position (advantage or disadvantage)


End result


win, loss or draw


Adv position ending in draw receives a score of
near zero


Dis
-
adv ending in a draw will receive better than
0.5






45

KRRKR Endgames



Results


Expert
-
defined performed extremely well against
Random and Best
0


Evolved programs performed generally as well as
expert defined, sometimes better

Percent of favorable results in game outcomes

46

Main Experiment


KQRKQR


Most complex endgame
we worked with


Still theoretical draw


Highly position
dependant


“noisy”


Larger trees


2
officers


Queens


Easier to mate



47

KQRKQR Endgames
-

changes


Added


more “heavy” terminals (and
components)


Boolean
Is
-
Not
-
Mate
-
in
-
one
, most time consuming
but necessary


Boolean
Is
-
My
-
King
-
Not
-
Trapped


Not all king’s moves lead closer to edges


Important but vague


usually happens with complex
terminals


My
-
Officers
-
Same
-
Line

48







Function Set

Complex Terminals

Simple Terminals

If3(B,F,F)

EvaluateMaterial

NotMyKingInCheck

Or2(B,B)

IsMaterialIncrease

IsOppKingInCheck

Or3(B,B,B)

IsMate

MyKingDistEdges

And2(B,B)

IsMateInOne

OppKingProximityToEdges

And3(B,B,B)

OppPieceCanBeCaptured

NumMyPiecesNotAttacked

Smaller(B,B)

MyPieceCannotBeCaptured

NumOppPiecesAttacked

Not(B)

IsOppKingStuck

ValueMyPiecesAttacking

IsMyKingNotStuck

ValueOppPiecesAttacking

IsOppKingBehindPiece

IsMyQueenNotAttacked

IsMyKingNotBehindPiece

IsOppQueenAttacked

IsOppPiecePinned

IsMyFork

IsMyPieceNotPinned

IsOppNotFork

NumMovesMyKing

NumNotMovesOppKing

MyKingProxRook

OppKingDistRook

MyPiecesSameLine

OppPiecesNotSameLine

IsOppKingProtectingPiece

IsMyKingProtectingPiece

Genome Summary

49

KQRKQR Endgames



New Opponent


CRAFTY
, second in the 2004 Computer Chess
Championship (held at Bar
-
Ilan)


Uses brute force methods; State
-
of
-
the
-
art
search algorithms


Specializes in Blitz games (typically 3 minutes
per game)


We limited to 5 secs per move, enough to
scan ~1.5 Million boards
with pruning

50

KQRKQR Endgames



Our parameters


Used lookahead of depth
2


Typically ~
5
secs per move


Simple Minimax search, but
not

Alpha
-
Beta


Played
5
-
6
moves per game


Never

cancelled a game, even if it started
with mate
-
in
-
4
(which CRAFTY easily saw)


Played each position
2
times, switching places


~
100
games
-

reduce noises in starting positions

51

Results: Master

52

Results: CRAFTY

53

Multiple Endgames


Aim for general
-
purpose strategies


All

endgames used during evolution


Results:


%Draws

%Advs

%Wins

68

2

6

Master

72

4

2

CRAFTY

54

Sample GP
-
Endchess

Tree
0
:

(If
3
(Or
2
(Not (Or
2
(And
2
OppPieceAttUnprotected NotMyKingInCheck) (Or
2
NotMyPieceAttUnprotected
100
*Increase)))
(And
2
(Or
3
(And
2
OppKingStuck NotMyPieceAttUnprotected) (And
2
OppPieceAttUnprotected OppKingStuck) (And
3
-
1000
*MateInOne OppKingInCheckPieceBehind NotMyKingStuck)) (Or
2
(Not NotMyKingStuck) OppKingInCheck)))
NumMyPiecesUNATT (If
3
(< (If
3
(Or
2
NotMyPieceAttUnprotected NotMyKingInCheck) (If
3
NotMyPieceAttUnprotected
#NotMovesOppKing OppKingInCheckPieceBehind) (If
3
OppKingStuck OppKingInCheckPieceBehind
-
1000
*MateInOne))
(If
3
(And
2 100
*Increase
1000
*Mate?) (If
3
(< NumMyPiecesUNATT (If
3
NotMyPieceAttUnprotected
-
1000
*MateInOne
OppKingProxEdges)) (If
3
(< MyKingDistEdges #NotMovesOppKing) (If
3
-
1000
*MateInOne
-
1000
*MateInOne
NotMyPieceATT) (If
3 100
*Increase #MovesMyKing OppKingInCheckPieceBehind)) NumOppPiecesATT) (If
3
NotMyKingStuck
-
100.0
OppKingProxEdges))) (If
3
OppKingInCheck (If
3
(Or
2
NotMyPieceAttUnprotected
NotMyKingInCheck) (If
3
(< MyKingDistEdges #NotMovesOppKing) (If
3
-
1000
*MateInOne
-
1000
*MateInOne
NotMyPieceATT) (If
3 100
*Increase #MovesMyKing OppKingInCheckPieceBehind)) NumOppPiecesATT) (If
3
(And
3
-
1000
*MateInOne NotMyPieceAttUnprotected
100
*Increase) (If
3
(< NumMyPiecesUNATT (If
3
NotMyPieceAttUnprotected
-
1000
*MateInOne OppKingProxEdges)) (If
3
(< MyKingDistEdges #NotMovesOppKing) (If
3
-
1000
*MateInOne
-
1000
*MateInOne NotMyPieceATT) (If
3 100
*Increase #MovesMyKing OppKingInCheckPieceBehind))
NumOppPiecesATT)
-
1000
*MateInOne)) (If
3
(< (If
3 100
*Increase MyKingDistEdges
100
*Increase) (If
3
OppKingStuck
OppKingInCheckPieceBehind
-
1000
*MateInOne))
-
100.0
(If
3
(And
2
NotMyPieceAttUnprotected
-
1000
*MateInOne) (If
3
(<
NumMyPiecesUNATT (If
3
NotMyPieceAttUnprotected
-
1000
*MateInOne OppKingProxEdges)) (If
3
(< MyKingDistEdges
#NotMovesOppKing) (If
3
-
1000
*MateInOne
-
1000
*MateInOne NotMyPieceATT) (If
3 100
*Increase #MovesMyKing
OppKingInCheckPieceBehind)) NumOppPiecesATT) (If
3
OppPieceAttUnprotected NumMyPiecesUNATT MyFork)))))

Tree
1
:

(If
3
NotMyPieceAttUnprotected #NotMovesOppKing
1000
*Mate?)

Tree
2
:

(If
3 1000
*Mate? NumMyPiecesUNATT
-
1000
*MateInOne)

55

Summary



Draw and better against Master
-
defined


Draw against a world class opponent


On limited conditions (not a full game, time ,etc.)


Shows deep search may have an alternative


Fast, pattern
-
oriented search suggests more
human resemblance


Search and lookahead are still important

56

Future Work


Add more pieces


Improve evolution speed


Parallel nets


Stronger board representations


Develop more cognitive models using evolution


Search scheme space as well as game space


Tackle beyond endgames


Openings and mid
-
game


General game concept schemas (?)