Representation and processing of novel stimuli by an associative neural network

prudencewooshΤεχνίτη Νοημοσύνη και Ρομποτική

19 Οκτ 2013 (πριν από 3 χρόνια και 11 μήνες)

106 εμφανίσεις

Representationandprocessingofnovel
stimulibyanassociativeneuralnetwork
ReimerKuhn
DepartmentofMathematics,King'sCollegeLondon
incollaborationwith
ValeriadelPrete
http://www.mth.kcl.ac.uk/staff/r
kuehn.html
Overview
Motivation
TheModel
CollectiveProperties
{FiringRateDistributions
{Stimuli,MutualInformation,Correlation

Perspective:
{OptimiseRepresentations
{Dynamics
Motivation
Representationofstimuliinanassociativeneuralnetworks
|neuralringpatterns,ratesandtheirdistributions
Ofinterest:dierencesinringratedistributions,ifnew
stimulusiscorrelatedwithpre-learntpatternsornot.
Fornewstimulicorrelatedwithpre-learntpatterns:
{mutualinformationbetweenringratesandnovelstimulus
{dependenceoncorrelationwithpre-learntpatterns
{dependenceonotherparameters
(numberofmemories,thresholds,neuralgain-function)
Theoreticalframeworkforinterpretationofrecordings
usingtrained-untrainedscenarios?
{untrainedanimalsrepresentstimuliinexistingcognitive
structure
{coginitivestructurechangesinresponsetonewstimuli
Results{atleastinprinciple{experimentallyaccessible
Context
ATrevesetal(NeuralComputation,1999)
Recordingsfrominferiortemporalcortex,visualstimuli
rate-distributionsnon-exponential,tstoassumedcurrent
distributions
NBrunel(J.Comp.Neurosci.,2000)
DynamicsofsparselyconnectednetworksofleakyIFneurons
withuniformsynapticstrengths:studyofcollectivenetwork
states
JHertzetal(Neurocomputing,2003);q-bio.NC/0402023
DynamicsofsparselyconnectednetworksofleakyIFneurons
withuniformsynapticstrengths:computationofFanofactors
TheModel
Gradedresponseneurons,Kirchhoequationsforcoupledleaky
integrators
Ci
dUi
dt
=
Ui
Ri
+
N
X
j=1
Jij
j
+Ii
Firingratesi
viavoltage-to-ratetransduction-function
i
=g(Ui
#i)
SynapticcouplingsfromHebbiancovariancelearningrule
Jij
=
1
Na(1a)
p
X
=1(

i
a)(
j
a)

i
=
8><>:
1;withprob:a;
0;withprob:1a;
hence:h

i
i=a
Voltage-to-ratetransduction-functionforpresentsetup:
g(x)=max
x
U0
+x
(x)
−202468
x
0
0.2
0.4
0.6
0.8
1
g(x)
Voltage-to-ratetransduction-function,x=U#,U
0
=0:75,max
=1.
Presenttalk:
onlylongtimestationaryresponse
Symmetriccouplings:dynamicsgovernedbyLyapunovfunction
HN
()=
1
2
N
X
i;j=1
Jij
i
j
+
N
X
i=1
G(i)
N
X
i=1(Ii
#i)i
with
G()=
Z

d
0g
1
(
0)
=)stationaryresponsefrom
minima
ofHN
Characterisationofattractors(minimaofHN
):
T!0-limitoffreeenergycorrespondingtoHN
(RK,S.BosJ.Phys.A,1993)
CollectiveProperties

Stationary
limitfromequilibriumstatisticalmechanics.
{Partitionfunction
ZN
=
Z
Y
i
di
exp[HN
()]
{Freeenergy
fN
()=(N)1
logZN
{T!0,!1-limit:onlyminimaofHN
contribute.
{Atechnicalpoint:
randomnessduetof

i
g
)fN
()
{Macroscopiccharacterizationofsystem|orderparameters:
m
=
1
N
X
i


i
a
a(1a)
hi
i
;q=
1
N
X
i
hi
i2
;c=

N
X
i
[h
2
i
i
hii2
]

Self-consistency
equations,T=0-limit:
m=
DD
a
a(1a)
^
EE;z;I
c=
1
p
r
DDz^
EE;z;I
r=
q
(1c)2
q=
DD^
2EE;z;I
with
^=^(;z;I)=gm(a)+
p
rz+
c
1c
^+I#
()
Inputcurrents
(stimuli)
Gaussian:Ii
=i
+i
withi
N(0;1)

Firingratedistribution
,parameterisedbym,c,q;from(*):
p(;I;)=
1
N
X
i
;i
(IIi
)
D(i)E
=
D(^)EI;
OrderParameters
00.10.20.3

0
0.05
0.1
0.15
0.2
0.25
0.3
m
no stimulus
=0.005
=0.010
=0.020
=0.030
Overlapasfunctionofloadinglevel,withoutstimulus,andfor=0:005,
=0:01,=0:02,=0:03,and=0:03.
OrderParameters
00.10.20.3

0
0.01
0.02
0.03
0.04
0.05
q
no stimulus
=0.005
=0.010
=0.020
=0.030
00.10.20.3

0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
c
no stimulus
=0.005
=0.010
=0.020
=0.030
Spin-glassorderparameterqandsusceptibilitycasfunctionsofloadinglevel,
withoutstimulus,andfor=0:005,=0:01,=0:02,=0:03,and=0:03.
FiringRateDistributions
−0.100.10.20.30.40.50.6

0
5
10
15
20
25
p()
no stimulus
=0.005
=0.010
=0.020
=0.030
Firingratedistributionat=0:005,withoutstimulus,andfor=0:005,
=0:01,=0:02,=0:03,and=0:03.
FiringRateDistributions
−0.100.10.20.30.40.50.6

0
5
10
15
20
25
p()
no stimulus
=0.005
=0.010
=0.020
=0.030
Firingratedistributionat=0:075,withoutstimulus,andfor=0:005,
=0:01,=0:02,=0:03,and=0:03.
FiringRateDistributions
−0.100.10.20.30.40.50.6

0
5
10
15
20
25
p()
no stimulus
=0.005
=0.010
=0.020
=0.030
Firingratedistributionat=0:10,withoutstimulus,andfor=0:005,
=0:01,=0:02,=0:03,and=0:03.
FiringRateDistributions
−0.100.10.20.30.40.50.6

0
5
10
15
20
25
p()
no stimulus
=0.005
=0.010
=0.020
=0.030
Firingratedistributionat=0:15,withoutstimulus,andfor=0:005,
=0:01,=0:02,=0:03,and=0:03.
PerformanceMeasures
Mutualconditionalinformationofringratedistributionand
inputcurrents
~
I(;Ij)=
X

Z
ddIp(;I;)log
2
p(;Ij)
p(j)p(Ij)
=
X

p()
Z
ddIp(jI)p(Ij)log2
p(jI;)
p(j)
Normalisedcorrelationbetweenringratesandcurrents
CI;
=
1
q

2


2
I
hhIihihIii
MutualInformationandCorrelation
00.050.10.150.20.25

0
0.2
0.4
0.6
0.8
1
I(,I|)
CI,
00.050.10.150.20.25

0
0.2
0.4
0.6
0.8
1
I(,I|)
CI,
Mutualconditionalinformationandcurrent-ratecorrelationat=0:03and
=0:005(left)and=0:01(right)asfunctionsof.
MutualInformationandCorrelation
00.050.10.150.20.25

0
0.2
0.4
0.6
0.8
1
I(,I|)
CI,
00.050.10.150.20.25

0
0.2
0.4
0.6
0.8
1
I(,I|)
CI,
Mutualconditionalinformationandcurrent-ratecorrelationat=0:03and
=0:02(left)and=0:03(right)asfunctionsof.
MutualInformation
0.000.050.100.15

0.0
0.2
0.4
0.6
0.8
1.0
=0.005
=0.01
=0.02
=0.03
Mutualconditionalinformationasafunctionofforvariousdegreesof
correlation.
SummaryandOutlook
Computedringratedistributionsinanalogueneuronsystems
Studieddependenceondegreeofcorrelationbetweenstimulus
andpre-learntpatterns
Startedsystematicevaluationofinformationtheoreticperfor-
mancemeasures
Results{atleastinprinciple{experimentallyaccessible
Usefulastheoreticalframeworkforinterpretationofrecordings
usingtrained-untrainedscenarios?

Withinreachand/ortobedone
{Gradedpatternsinthelearningrule
{Patterndistributionthatmaximisesmutualinformation?
{Asymmetriccouplings
{Non-stationaryeects,dynamics)J.Hatchett(KCL)
{Morerealisticneuronmodels