Evolutionary Computations, Genetic Rule-based Systems, and Evolutionary Games for Real-word and Military Applications

boorishadamantΤεχνίτη Νοημοσύνη και Ρομποτική

29 Οκτ 2013 (πριν από 3 χρόνια και 7 μήνες)

71 εμφανίσεις

Evolutionary Computations, Genetic
Rule
-
based Systems, and
Evolutionary Games for Real
-
word
and Military Applications


Jae C. Oh, Ph.D.

EECS

Syracuse University

Overview of Presentation

1.
Evolutionary Computations (ECs)


Some verities


I.e., GA, ES, EP, GP, etc.


The Main Idea of ECs

2.
Genetic Learning Classifier Systems


Speaker Identification


Other possible applications

3.
Multi
-
agent Systems and Evolutionary Game
theory, Emergent Behavior


Applications



I should go to the game


Brief History


Evolutionary Programming


Fogel in 1960s


Individuals are encoded to be finite state
machines


Intelligent Behavior


Evolutionary Strategies


Rechenberg, Schwefel in 1960s


Real
-
valued parameter optimization


Genetic Algorithms


Holland in 1960s


Adaptive Systems


Crossover Operators


Present Status


Wide variety of evolutionary algorithms


No one seriously tries to distinguish them except
for some special cases


Mostly for theoretical reasons




We will call all Evolutionary Algorithms




And I will call them Genetic Algorithms or
Evolutionary Algorithms, interchangeably.

Evolution is a search process

From the Tree of the Life Website,

University of Arizona

Orangutan

Gorilla

Chimpanzee

Human

Evolution is a parallel search

AAGACTT

AGGACTA

TG
GACTT

AAG
G
C
C
T

T
GGACTA

AG
T
GAC
C
A

A
G
GGC
A
T

T
AG
C
CCT

A
G
C
ACTT

AG
G
G
CA
A

CAGCA
CCA

A
G
C
ACTA

TAGCCC
A

TAG
A
C
T
T

AGC
G
CTT

AGCAC
AA

AGGGCAT

TC
GCCCA

T
AG
GC
C
T
A

AG
TG
CTA

AG
T
AC
A
A

A
A
GGCAA

What are

Evolutionary

Algorithms?


Find solutions for a problem using the idea of
evolution:




Randomized
search and optimization
algorithms

guided by the principle of
Darwin’s natural selection:
Survival of
fittest.



Evolve potential solutions


Search!!


Evolutionary Algorithms?


Search Algorithms?


Learning Algorithms?


Function Optimization Algorithms?



They are fundamentally the same!!

Search

Search

Search

Notion of Search Space


Real world problem


Search space


Abstraction
-
> State Space


Exploring the state space for given problem


Search Algorithms


My friend

Search Space

Dome

Genetic Algorithm in search space

1.
Starting with a subset of
n

randomly chosen

solutions ( ) from the search space (i.e.

chromosomes). This is the
population

2.
This population is used to produce a next
generation

of individuals by reproduction

3.
Individuals with a higher
fitness


(i.e., smaller |
-

|)have more chance to
reproduce (i.e. natural selection)



The one

GA in Pseudo code

0 START

: Create random population of
n

chromosomes

1 FITNESS :
Evaluate fitness
f(x)

of each chromosome in



the population


2
NEW POPULATION

1
SELECTION :
Based on
f(x)

2
RECOMBINATION

:
Cross
-
over chromosomes

3
MUTATION :
Mutate

chromosomes

4
ACCEPTATION :
Reject or accept new one

3
REPLACE :
Replace old with new population:


the new generation

4 TEST :
Test problem criteria

5 LOOP :
Continue step 1


4 until criteria is




satisfied

Learning Algorithms


Finding (through search) a suitable program,
algorithm, function for a given problem

Learning Algorithm

Training Data

(Experience)

Program

Learning Algorithms (function Optimizations)

Problem instance

Hypothesis Space

Program Space

Function Space

The One??

Set of Hypothesis

Learning Algorithms (Digression)


How do we know the found hypothesis,
program, function, etc. are
the one

we are
looking for?


We don’t know for sure


Is there any mathematical way of telling how good
hypothesis is?


I.e., |h(x)


f(x)| = ?


Computational Learning Theory can tell us this


Valiant (1984)

Genetic Learning Classifier Systems


General Learning Systems [Russell & Norvig]

Critic

Learning Element

Problem Generator

Performance Element

Environment

Sensors

changes

Knowledge

Learning Goals

Feedback

Actuators

Performance Standard

Genetic Learning Classifier Systems


Use population of rules


Rules: if <condition>
n

then <action>
m

Detector

Effector

Genetic Algorithm

Conditions
-
> Actions

Sensors

Learning Element

Actuators

Performance Element

Problem Generator

Critiic

Environment

Rule
-
based Learning Classifier Systems


Two approaches


The Michigan Approach [Riolo, Holland]


The Pitt Approach [Smith]


Rules are represented with 0, 1, #


0: absence of info/knowledge


1: existence of info/knowledge


#: don’t care condition.


Rules are randomly generated


Evolved using GA

Example (Wolf or Grandmother?)

0

1

0

1

#

0

0

1

0

GrandMa

kind

ears

num. of legs

smart

scream

runaway

kiss

teeth

Encoding

1

0

1

1

#

1

1

0

1

Wolf

kind

ears

num. of legs

smart

scream

runaway

kiss

teeth

Encoding

Detector

Effector

Genetic Algorithm (Learning)

Teeth, kind, ears, num
-
legs, smart
-
> scream, runaway, kiss

0, 1, #, # 1, 1
-
> 0, 0, 0

0, 1, #, 1 1, 0
-
> 1, 0, 0

0, 1, 1, # 1, 0
-
> 0, 0, 1

Application to Speaker Identification

Communications (tend to be short and “bursty”)

Application to Speaker Identification


Text
-
independent or Text
-
dependent?


Closed
-
set or Open
-
set?


The system must be robust to noise


Traditional statistical methods works well for the
closed
-
set problem


Neural
-
net


But the open
-
set problem is challenging


Need the ability to introduce new speakers
dynamically



Use of Rule
-
based Genetic Classifier System With the Audio Group at AFRL (Grieco)

An Overview of Speaker Identification

Training Phase

Model
training

Lt. Oh

Feature
extraction

Database of
Codebooks for
each speaker

Model

training

Lt. Oh

Lt. Oh

Decision

Accepted!

Speaker
Model

Feature
extraction

Testing/Identification Phase

Decision

Voice Input

Threshold

Lt. Oh

GA Rule
-
based Speaker ID System

codebook

Effector

Genetic Algorithm (Learning)

Codebooks
-
> Actions

Temporal loop

<codebook1, … codebook N>
-
> <Lt. Oh>

<codebook1, … codebook N>
-
> <Smith>

Other Applications of GA Rule
-
based Systems


Missile Evasion Problem [Shultz, Grenfenstte, 1995]



UAV or Autonomous Robots [NRL, AFRL]:


Simple rules.


Learning to follow


Learning to make teams


Flocks around


What’s different? Emergent Behavior, Interactions among
agents



Handwritten Character recognition [Oh 1993]



Learning Classifier Web page:
http://www.cs.bath.ac.uk/~amb/LCSWEB/


Multi
-
agent Systems and Game Theory


Agents: Selfishly Maximizing Utility



Do the right thing

but information is limited



Goals of individual agents can conflict.



Utility of an agent can depends on what others are
doing too



The action(s) that maximizes the
Expected Value of
the Performance Measures

(= Utility) given the
percept sequence to date.



Being rational doesn’t mean the smartest!

Multi
-
Agent Systems Research


the study, construction, and application of multi
-
agent systems, involve several
interacting
, intelligent
agents pursue some set of goals or perform some set
of tasks



Goal
-

and Task
-
Oriented Coordination: Cooperation,
Collaboration, and Competition




Micromotives and Macrobehavior






Thomas Schelling





W.W. Norton and Co. Inc

Interactions among agents


Things to consider:



Are all agents under the same administration
roof?


Are agents homogeneous or heterogeneous?


Are agents rational? sacrificing? Cooperating?
Collaborating?



Hierarchical? Distributed?

Areas of interests


Military Applications


Unmanned
-
Robots


Elimination of Mines (Coordination, Collaboration)


Search and Rescue


UAVs


Autonomous Sensor Arrays (Sensor dusts)


War games (Collaborating with 5
th

Generation war game
group. (Gemelli, Bello, & Wright)


Etc.


Industry


High
-
performance Distributed Computing


Replica Management


Resource sharing and allocation

Gnutella, GRID computing


Computer on
-
line games


E
-
commerce, etc.


Multi
-
agent systems and other
sciences

Social Sciences,

Biology, etc.

Computer Scientists

Create Theory and Models

Create Agents,

Computer s/w, etc.

Insects, Animals,

Humans, etc.

We need a formalism


Economic Game Theory


Von Neumann and
Morganstern: Theory of
Games and Economic Behavior


My interest


Promoting cooperation among rational agents


Prisoner’s Dilemma Game (PD)



Examples of

General


PD


Building a bridge: participate or not


Two competing companies setting the price
for a product: high price or low price


Nuclear Arms race (Mutually Assured
Destruction): build more bombs or not


Server replication


Coalition formation among rational agents


War game


Search and rescue by multiple parties


Many many others


Classic Prisoner’s Dilemma Game


Two accomplices in bank robberies caught by the
Police; Interrogated separately; The police are
bargaining


Choice of a prisoner:
Confess

(
Defect
) or
Do not
confess

(
Cooperate
) to the other prisoner


Dominant Strategy is to defect


Prisoner One

Prisoner Two

Payoff Matrix for 2
-
Person PD

C

D

C

R/R (3/3)

S/T (0/5)

D

T/S

(5/0)

P/P (1/1)

T > R > P > S and 2R > T + S,

Typically, T = 5, R = 3, P = 1, and S = 0

Prisoner 1

Prisoner 2

Pareto
-
Optimal

Nash

Equilibrium

Tragedy of the Commons

Versus



Commons” is Resource Shared



Agents are selfishly rational

In Repeated PD Games


Defect is not an obvious choice for winning
strategy



Promoting cooperation is nevertheless
difficult. But very important!!


When there are many many agents


Central coordination is too expensive


Let them survive by interacting locally


And globally exhibit desired emergent behavior

Rational Agents (No clustering incentive)

Rational Agents (Clustering incentive)

Some notable results


Given number of agents and resource amount,
is cooperation beneficial to me?


Lower
-

upper
-
bound on amount of resources
found, mathematically [Oh 2000].


Evolutionary Game can evolve strategies that
can promote cooperation and survive. [Riolo,
Oliphant, Axelrod, etc]

Conferences and Workshops


Congress on Evolutionary Computations


GECCO


Problem Solving in Nature


Hybrid Intelligent Systems


Frontiers in Evolutionary Algorithms


Applications of Game Theory and Artificial
Intelligence Techniques on Distributed
Computing and Internet
-
wide computing with
PDCS


Games and Emergent Behavior in Distributed
Computing Environments with PPSN



Conclusions


Historically, Evolutionary Computation came a
long way.


From single system learning/search/optimization
algorithms


To multi
-
agent systems


The application areas are numerous.


The time is for distributed systems (not only for
computers, but robots, sensors, people, soldiers,
etc.)


Evolutionary game theory and Classifier System
are very promising.