Applied to Intelligent Robotic Decision Making

rucksackbulgeAI and Robotics

Dec 1, 2013 (3 years and 7 months ago)

253 views

Computational Neuroscience, Vision and Acoustic Systems

Arlington, VA,
June 9, 2010

Phil Goodman
1,2
,
Fred Harris,
Jr

1,2

,

Sergiu Dascalu
1,2
,

Florian Mormann
3
& Henry Markram
4

1
Brain
Computation Laboratory, School of Medicine, UNR

2
Dept. of
Computer Science & Engineering
, UNR

3
Dept. of Epileptology, University of Bonn, Germany

4
Brain Mind Institute, EPFL, Lausanne, Switzerland



Large
-
Scale Biologically Realistic Models of Brain Dynamics

Applied to Intelligent Robotic Decision Making


ONR

N00014
-
10
-
1
-
0014



Graduate Students

Brain models & NCS


Laurence Jayet


Sridhar Reddy


Robotics


Sridhar Reddy


Roger Hoang




Cluster Communications


Corey Thibeault





Investigators

Fred
Harris, Jr
.

Sergiu Dascalu

Phil Goodman

Henry Markram

EPFL

Contributors


ChildBot

Florian Mormann

U Bonn

Mathias Quoy

U de Cergy
-
Pontoise

Neuroscience

Mesocircuit Modeling

Present Scope of Work


Robotic/Human
Loops

(Virtual Neurorobotics)

Software/Hardware
Engineering

Neuroscience

Mesocircuit Modeling

Robotic/Human
Loops

(Virtual Neurorobotics)

Software/Hardware
Engineering

Brain
slice technology
to Physiology

Neuroscience

Mesocircuit Modeling

Robotic/Human
Loops

(Virtual Neurorobotics)

Software/Hardware
Engineering

Neural Software Engineering

NCS is the only system with

a real
-
time robotic interface

(bAC)

K
AHP

800

excitatory

neurons

G
exc

P
connect

200

inhibitory

neurons

G
exc

P
connect

G
inh

P
connect

G
inh

P
connect

“Recurrrent Asynch Irreg Nonlinear” (RAIN) networks

RAIN Activity

HUMAN Wakeful RAIN Activity

ISI distrib

(10 min)

Rate

(cellwise)

CV (std/mn)

(cellwise)

(1 minute window)

R Parietal

5s close
-
up

Mesocircuit

RAIN: “Edge of Chaos”


Originally coined wrt cellular automata:
rules for complex processing most likely
to be found at “phase transitions” (PTs)
between order & chaotic regimes
(Packard 1988; Langton 1990; but
questioned by Mitchell et al. (1993)



Hypothesis here wrt Cognition, where
SNN have components of SWN, SFN,
and exponentially truncated power laws



PTs cause rerouting of ongoing activity
(OA), resulting in measured rhythmic
synchronization and coherence



The direct mechanism is not embedded
synfire chains, braids, avalanches, rate
-
coded paths, etc.



Modulated by plastic synaptic structures



Modulated by neurohormones (incl OT)



Dynamic systems & directed graph theory
> theory of computation


Edge of Chaos Concept


Lyapunov exponents on human unit simultaneous recordings

from Hippocampus and Entorhinal Cortex

Unpublished data, 3/2010: Quoy, Goodman

Neocortical
-
Hippocampal Navigation


A Circuit
-
Level Model of Hippocampal Place Field Dynamics

Modulated by Entorhinal Grid and Suppression
-
Generating Cells

Laurence C. Jayet
1*
, and Mathias Quoy
2
, Philip H. Goodman
1

1

University of Nevada, Reno

2

Université de Cergy
-
Pontoise, Paris


w/o K
ahp

channels


NO
intra
cellular theta precession


Asymm ramp
-
like depolarization


Theta power & frequ increase in PF

Explained findings of Harvey et al.

(2009) Nature 461:941

EC lesion


EC grid cells ignite PF


EC suppressor cells stabilize

Explained findings of Van Cauter et al.

(2008) EJNeurosci 17:1933

Harvey et al.

(2009) Nature 461:941

Neuroscience

Mesocircuit Modeling

Robotic/Human
Loops

(Virtual Neurorobotics)

Software/Hardware
Engineering

Sunfire X4600

GPU

Beowulf

200 cpu

Neuroscience

Mesocircuit Modeling

Robotic/Human
Loops

(Virtual Neurorobotics)

Software/Hardware
Engineering

Virtual Neuro
-
Robotics

Behavioral VNR System

Human trials using intranasal OT


Willingness to trust, accept social risk (Kosfeld 2005)


Trust despite prior betrayal (Baumgartner 2008)


Improved ability to infer emotional state of others (Domes 2007)


Improved accuracy of classifying facial expressions (Di Simplicio 2009)


Improved accuracy of recognizing angry faces (Champaign 2007)


Improved memory for familiar faces (Savaskan 2008)


Improved memory for faces, not other stimuli (Rummele 2009)


Amygdala less active & less coupled to BS and neocortex


w/ fear or pain stimuli (Kirsch 2005, Domes 2007, Singer 2008)


Oxytocin Physiology


Neuroanatomy


OT is 9
-
amino acid cyclic peptide


first peptide to be sequenced & synthesized! (ca. 1950)


means “rapid birth”: promotes uterine contraction


promotes milk ejection for lactation


reflects release from pituitary into the blood stream

“neurohypophyseal OT system”


rodents
: maternal & paternal bonding


voles
: social recognition of cohabitating partner vs stranger


ungulates
: selective olfactory bonding (memory) for own lamb


seems to modulate the saliency & encoding of sensory signals

“direct CNS OT system”

(OT & OTR KOs & pharmacology)


Inputs from neocortex, limbic system, and brainstem


Outputs:

Local dendritic release of OT into CNS fluid

Axonal inhib synapses in amygdala & NAcc



SON: magnocellular to pituitary


PVN: parvocellular to amygdala

& brainstem


axon to CNS

to PITUITARY

Magno

Parvo

fluid to CNS

“Trust & Affiliation” paradigm


Willingness to exchange token for food

Time spent facing

A
mygdala [fear response]:
inhibited by
HYp

oxytocin

HY
pothalamus
p
araventricular
nucleus [trust]: oxytocin neurons

Phase I: Trust the Intent (
TTI
)


Phase II: Emotional Reward Learning (
ERL
)

PR

VC

DPM

IT

oxytocin

VC

V
isual

C
ortex

PFdl

VPM

AC

A
uditory
C
ortex

AC

PFdl

P
refrontal,
D
orsolateral:

sustained suppression

PR

P
arietal
R
each
(LIP):

reach
decision making

V
entral
P
re
M
otor:

sustained activity

VPM

“Trust & Learn” Robotic Brain Project


D
orsal
P
re
M
otor:

planning
& deciding

DPM

BG

BG


B
asal
G
anglia:


decision
making

AM

AM

HYp

HYp

HPF

HPF


H
ippo
C


F
ormation

EC

HPF

EC


E
ntorhinal


C
ortex

I
nfero
T
emporal cortex:

responds to faces

IT

Phase I: Trust the Intent (
TTI
)

1.
Robot brain initiates
arbitrary sequence
of motions

2.
human moves object in
either
a similar
(“match”), or different
(“mismatch”)
pattern

Robot
Initiates Action

Human
Responds

LEARNING

Match:

robot
learns to trust

Mismatch:
don’t trust

3.
human
slowly
reaches for
an
object on the table

4.
Robot
either

trusts
”,
(assists/offers
the
object),
or

distrusts
”,
(retract

the
object).

Human
Acts

Robot
Reacts

CHALLENGE (at any time)

trusted

distrusted

Gabor V1
-
3 emulation

Phase II:
Emotional Reward Learning (
ERL
)

1.
human
initiates
arbitrary sequence
of object motions

Human Initiates Action

LEARNING

GOAL (after several + rewards)

Matches
consistently

2.
robot moves object in
either
a similar
(“
match”),
or different (“mismatch”)
pattern

Robot Responds

Match:

voiced +reward

Mismatch:
voiced

reward

Early ITI Results


Concordant > Trust

Discordant > Distrust

mean synaptic strength

Neuroscience

Mesocircuit Modeling

Robotic/Human
Loops

(Virtual Neurorobotics)

Scope of Work in the Coming Year


Software/Hardware
Engineering

Sunfire X4600

GPU

EC

CA

The Quad
at UNR