Water Resource Management

tentchoirAI and Robotics

Nov 15, 2013 (3 years and 6 months ago)

94 views

UNESCO Crossing the Chasm

Motivated Machine Learning for

Water Resource Management


Janusz Starzyk

School of Electrical Engineering and
Computer Science, Ohio University, USA


www.ent.ohiou.edu/~starzyk

UNESCO Workshop on Integrated
Modeling Approaches to Support
Water Resource Decision Making:
Crossing the Chasm

UNESCO Crossing the Chasm



Challenges in Water Management


Embodied Intelligence (EI)


Embodiment of Mind


EI Interaction with Environment


How to Motivate a Machine


Goal Creation Hierarchy


GCS Experiment


Promises of EI


To economy


To society

Outline

UNESCO Crossing the Chasm

Water management is challenging for various reasons:



Strategies in water management are developed
mostly on national level


There is a growing competition between countries
for water


Water policy making effects environment and
society, health and development, and economy


Growing demands of countries’ populations for
water


Leads to hydrological nationalism


Creates a need to integrate water sciences and policy
making


There is an acute need for legitimate scientific data


Decision making in water
-
related health, food and
energy systems are critical to economical
development and security

Challenges in Water
Management

UNESCO Crossing the Chasm

Decision makers must answer important questions:



How do we make water use sustainable?


Who owns the water?


What policies, institutional and legal framework
can promote sustainable use of water?


How to protect water resources from overuse
and contamination?


Water problems became too complex,
interconnected and large to be handled by any
one institution or by one group of professionals


It is a challenge to evolve strategies and
institutional frameworks for quick policy
changes towards an acceptable water use


It is necessary to create assessment and
modeling tools to improve policy making and
facilitate interaction.

Challenges in Water
Management

UNESCO Crossing the Chasm

Why development of integrated modeling to
support decision making is important ?



Computerized models were used for many
years to support water related decisions.


Models often simplify dynamics of economic,
social and environmental interactions and lead
to inappropriate policy making and
management decisions.


This note proposes models to emerge from
interaction with real dynamically changing
environments with all of their complexities and
societal dependencies.


The main objective is to suggest an integrated
modeling framework that may assist with the
tasks of water related decision making.

Challenges in Water
Management

UNESCO Crossing the Chasm

What are deficiencies of machine created models?



Artificial neural networks, fuzzy logic, and genetic
algorithms have all been used to model the hydrological
cycle


However, it is still difficult to apply these tools in making
real
-
life water decisions as the related parameters are not
explicitly known


What may be needed is a
motivated machine learning

for
characterizing the data and making predictions about their
dynamic changes


It could support intelligent decision making in dynamically
changing environment


It could be used to observe impacts of alternative water
management policies


It may consider social, cultural, political, economic and
institutional elements that influence decision making


This strategic note presents a
goal creation approach in
embodied intelligence

(EI) that motivates machine to
develop into a useful research toll.

Challenges in Water
Management

UNESCO Crossing the Chasm

Embodied intelligence may
support decision making:



EI mimics biological intelligent systems,
extracting
general principles of intelligent behavior and applying
them to design intelligent agents


It uses emerging, self
-
organizing,
goal creation

(GC)
system that motivates embodied intelligence to learn
how to efficiently interact with the environment



Knowledge is not entered into such systems, but rather
is a result of their successful use in a given environment.


This knowledge is validated through active interaction
with the environment.



Motivated intelligent systems adapt to unpredictable
and dynamic situations in the environment by learning,
which gives them a high degree of autonomy


Learning in such systems is incremental, with
continuous prediction of the input associations based
on the emerging models
-

only new information is
registered in the memory

Challenges in Water
Management

UNESCO Crossing the Chasm

Use the motivated learning scheme to integrate
modelling and decision making:



It is suggested to apply ML approach to water
management in changing environments where the
existing methods fail or work with difficulty.


For

instance,

using

classical

machine

learning

to

predict

the

future

for

physical

processes

works

only

under

the

assumption

that

same

processes

will

repeat
.


However,

if

a

process

changes

beyond

certain

limits,

the

prediction

will

not

be

correct
.



GC

systems

may

overcome

this

difficulty

and

such

systems

can

be

trained

to

advice

humans
.


Design concepts, computational mechanisms, and
architectural organization of embodied intelligence
are presented in this talk


The talk will explain internal motivation mechanism
that leads to effective goal oriented learning


In addition, a goal creation mechanism and goal
driven learning will be described.

Challenges in Water
Management

UNESCO Crossing the Chasm


“…
Perhaps the last frontier of science


its
ultimate challenge
-

is to understand the biological
basis of consciousness and the mental process by
which we perceive, act, learn and remember..”

from
Principles of Neural Science by
E. R. Kandel et al.


E. R. Kandel won Nobel Price in 2000 for his work on physiological
basis of memory storage in neurons.


“…
The question of intelligence is the last great
terrestrial frontier of science...”

from Jeff Hawkins
On
Intelligence.



Jeff Hawkins founded the Redwood Neuroscience Institute devoted
to brain research



Intelligence

AI’s holy grail

From

Pattie Maes MIT Media Lab

UNESCO Crossing the Chasm

Traditional AI



Embodied Intelligence


Abstract intelligence


attempt to simulate
“highest” human faculties:


language, discursive
reason, mathematics,
abstract problem solving


Environment model


Condition for problem
solving in abstract way



“brain in a vat”



Embodiment


knowledge is implicit in the
fact that we have a body


embodiment supports brain
development


Intelligence develops
through interaction with
environment


Situated in environment


Environment is its best model


UNESCO Crossing the Chasm

Design principles of intelligent systems

from Rolf Pfeifer “Understanding of Intelligence”, 1999



Interaction with
complex environment


cheap design


ecological balance


redundancy principle


parallel, loosely
coupled processes


asynchronous


sensory
-
motor


coordination


value principle


Agent

Drawing by Ciarán O’Leary
-

Dublin Institute of Technology

UNESCO Crossing the Chasm

Embodied Intelligence


Definition


Embodied Intelligence (EI) is a
mechanism that learns
how to survive in a hostile environment



Mechanism:

biological, mechanical or virtual agent


with embodied sensors and actuators


EI acts on environment and perceives its actions


Environment hostility is persistent and stimulates EI to act


Hostility:

direct aggression, pain, scarce resources, etc


EI learns so it must have associative self
-
organizing memory


Knowledge is acquired by EI

UNESCO Crossing the Chasm

Embodiment
Actuators
Sensors
Intelligence
core
channel
channel
Embodiment
Sensors
Intelligence
core
Environment
channel
channel
Actuators
Embodiment
Actuators
Sensors
Intelligence
core
channel
channel
Embodiment
Sensors
Intelligence
core
Environment
channel
channel
Actuators
Embodiment of a Mind



Embodiment contains
intelligence core and
sensory motor interfaces
under its control to interact
with environment


Necessary for development
of intelligence


Not necessarily constant or
in the form of a physical
body


Boundary transforms
modifying brain’s self
-
determination

UNESCO Crossing the Chasm


Brain learns own body’s dynamic


Self
-
awareness is a result of
identification with own embodiment


Embodiment can be extended by
using tools and machines


Successful operation is a function
of correct perception of
environment and own embodiment


Embodiment of a Mind

UNESCO Crossing the Chasm

INPUT

OUTPUT

Simulation or

Real
-
World System

Task

Environment

Agent Architecture

Long
-
term Memory

Short
-
term Memory

Reason

Act

Perceive

RETRIEVAL

LEARNING

EI Interaction with Environment

From Randolph M. Jones, P : www.soartech.com

UNESCO Crossing the Chasm

How to Motivate a Machine

?


The fundamental question is how to
motivate a machine to do anything, in
particular to increase its “brain”
complexity?


How to motivate it to explore the
environment and learn how to
effectively work in this environment?


Can a machine that only implements
externally given goals be intelligent?

If not how these goals can be
created
?



UNESCO Crossing the Chasm





I suggest that hostility of environment motivates us
.


It is the pain that moves us.


Our intelligence that tries to minimize this pain motivates our actions,
learning and development


We need both the environment hostility and the mechanism
that learns how to reduce inflicted by the environment pain


How to Motivate a Machine

?




In this work I propose, based on the
pain, mechanism that motivates the
machine to act, learn and develop.


So the pain is good
.


Without the pain there will be no intelligence
.


Without the pain there will be no motivation to
develop
.

UNESCO Crossing the Chasm

Pain
-
center and Goal Creation


Simple Mechanism


Creates hierarchy of
values


Leads to formulation of
complex goals


Reinforcement :


Pain increase


Pain decrease


Forces exploration


+

-

Environment

Sensor

Motor

Pain level

Dual pain level


Pain increase

Pain decrease

(
-
)

(+)

Excitation

(
-
)

(
-
)

(+)

(+)

Wall
-
E’s
goal is to keep

his plants from dying

UNESCO Crossing the Chasm

Primitive Goal Creation

-

+

Pain

Dry soil

Primitive

level

open

tank

sit on

garbage

refill

faucet

w. can

water

Dual pain

UNESCO Crossing the Chasm

Abstract Goal Creation



The goal

is to reduce
the primitive pain level



Abstract goals

are
created to reduce
abstract
pains

in order to satisfy the
primitive goals



Abstract pain center


-

+

Pain

Dual pain

+

Dry soil

Abstract pain


water can”


sensory input
to abstract pain
center

Sensory pathway

(perception, sense)

Motor pathway

(action, reaction)

Primitive
Level

Level I

Level II

faucet

-

w. can

open

water

Activation

Stimulation

Inhibition

Reinforcement

Echo

Need

Expectation

UNESCO Crossing the Chasm

Abstract Goal Hierarchy



A hierarchy of
abstract goals

is
created
-

they satisfy

the lower level goals

Activation

Stimulation

Inhibition

Reinforcement

Echo

Need

Expectation

-

+

+

Dry soil

Primitive
Level

Level I

Level II

faucet

-

w. can

open

water

+

Sensory pathway

(perception, sense)

Motor pathway

(action, reaction)

Level III

tank

-

refill

UNESCO Crossing the Chasm

GCS vs. Reinforcement Learning

Environment
Critic
States
Value
Function
Policy
reward
action
Environment
Critic
States
Value
Function
Policy
reward
action
RL Actor
-
critic design

Goal creation system

Case study: “How can
Wall
-
E

water his plants if
the water resources are limited and hard to find?”

Sensory

pathway

Motor

pathway

GCS

Environment

Pain

States

Gate control

Desired

action

&state

Action

decision

Action

UNESCO Crossing the Chasm

Goal Creation Experiment

Sensory
-
motor pairs and their effect on the environment


PAIR #

SENSORY

MOTOR

INCREASES

DECREASES

1

water can

water the plant

moisture

water in can

8

faucet

open

water in can

water in tank

15

tank

refill

water in tank

reservoir water

22

pipe

open

reservoir water

lake water

29

rain

fall

lake water

-

UNESCO Crossing the Chasm

Results from GCS scheme

0

100

200

300

400

500

600

0

2

4

pain

Dry soil

0

100

200

300

400

500

600

0

1

2

pain

No

water in can

0

100

200

300

400

500

600

0

1

2

pain

No water in tank

0

100

200

300

400

500

600

0

0.5

1

pain

No water in reservoir

0

100

200

300

400

500

600

0

2

4

pain

No water in lake

UNESCO Crossing the Chasm

Averaged performance over 10 trials:

GCS:

RL:

0
100
200
300
400
500
600
0
0.5
1
pain
Primitive pain
0
100
200
300
400
500
600
0
0.5
1
pain
Lack of food
0
100
200
300
400
500
600
0
0.2
0.4
pain
Lack of money
0
100
200
300
400
500
600
0
0.2
0.4
pain
Lack of bank savings
0
100
200
300
400
500
600
0
0.2
0.4
pain
Lack of job opportunity
0
100
200
300
400
500
600
-1
0
1
pain
Lack of school opportunity
Machine using GCS learns to control all abstract pains and
maintains the primitive pain signal on a low level in
demanding environment conditions.


0

100

200

300

400

500

600

0

10

20

30

GCS vs. Reinforcement Learning

UNESCO Crossing the Chasm

Goal Creation Experiment

Action scatters in 5 CGS simulations

0
100
200
300
400
500
600
0
5
10
15
20
25
30
35
40
Goal Scatter Plot
Goal ID
Discrete time
UNESCO Crossing the Chasm

Goal Creation Experiment

The average pain signals in 100 CGS simulations

0

100

200

300

400

500

600

0

0.5

Primitive pain


dry soil

Pain

0

100

200

300

400

500

600

0

0.1

0.2

Lack of water in can

Pain

0

100

200

300

400

500

600

0

0.1

0.2

Lack of water in tank

Pain

0

100

200

300

400

500

600

0

0.1

0.2

Lack of water in reservoir

Pain

0

100

200

300

400

500

600

0

0.05

0.1

Lack of water in lake

Pain

Discrete time

UNESCO Crossing the Chasm

Compare RL (TDF) and GCS

Mean primitive pain
Pp value as a
function of the
number of
iterations.



Dashed lines indicate
moment when Pp is
getting stable

-

green line for TDF

-

blue line for GCS.

UNESCO Crossing the Chasm


Comparison of
execution time on
log
-
log scale


TD
-
Falcon green


GCS blue



Combined
efficiency of GCS
1000 better than
TDF



Compare RL (TDF) and GCS

Problem solved

Conclusion:
embodied intelligence, with motivated learning based on
goal creation system, effectively integrates
environment
m
odeling
and decision making



thus it is poised to cross the chasm

UNESCO Crossing the Chasm

Promises of embodied intelligence


To society


Advanced use of technology


Robots


Tutors


Intelligent gadgets


Intelligence age follows


Industrial age


Technological age


Information age


Society of minds


Superhuman intelligence


Progress in science


Solution to societies’ ills


To industry


Technological development


New markets


Economical growth

ISAC, a Two
-
Armed Humanoid Robot

Vanderbilt University


UNESCO Crossing the Chasm

2002

2010

2020

2030

Biomimetics and Bio
-
inspired Systems

Impact on Space Transportation, Space Science and Earth Science

Mission Complexity

Biological Mimicking

Embryonics

Extremophiles

DNA

Computing

Brain
-
like

computing

Self Assembled Array

Artificial nanopore

high resolution

Mars in situ

life detector

Sensor Web

Skin and Bone

Self healing structure

and thermal protection

systems

Biologically inspired

aero
-
space systems

Space Transportation

Memristors

Biological nanopore

low resolution

UNESCO Crossing the Chasm

Sounds like science fiction


If you’re trying to look far
ahead, and what you see
seems like science fiction,
it might be
wrong
.


But if it

doesn’t seem
like
science fiction, it’s

definitely wrong.

From presentation by Foresight Institute

UNESCO Crossing the Chasm

Questions?