Introduction to Neural Networks Abstract - ima

prudencewooshAI and Robotics

Oct 19, 2013 (3 years and 9 months ago)

95 views

Introduction to Neural Networks
Introduction to Neural Networks
Navapat Khantonthong
Navapat Khantonthong
n nz zk k@ @c cs s..n no ot tt t..a ac c..u uk k
R Ro oo om m B B3 36 6
S Su up pe er rv viis so or rs s: :
Prof. Uwe Aickelin
Prof. Uwe Aickelin
Dr. Julie Greensmith
Dr. Julie Greensmith
Abstract
Abstract
 Neural Network is the representation of brain's
Neural Network is the representation of brain's
learning approach. This brain operates as
learning approach. This brain operates as
multiprocessor and has excellent interlinked. Neural
multiprocessor and has excellent interlinked. Neural
N Ne et tw wo or rk k a alls so o c ca an n b be e r re ep pr re es se en nt te ed d a as s " "P Pa ar ra alllle ell
d diis st tr riib bu ut te ed d p pr ro oc ce es ss siin ng g" " p plla an nn niin ng g.. I It t iis s u ut tiilliis se ed d iin n t th he e
computer applications for solving the complicated
computer applications for solving the complicated
problems. There are many benefits from Neural
problems. There are many benefits from Neural
Network such as no requirements for specifying the
Network such as no requirements for specifying the
r re elle ev va an nt t ffa ac ct to or rs s,, a an n u un ns so op ph hiis st tiic ca at te ed d m mo od de ell w wh hiic ch h h ha as s
m ma an ny y ffa ac ct to or rs s ffo or r p pe er rffo or rm ma an nc ce e,, a a s st tr ra aiig gh ht tffo or rw wa ar rd d
model, fault tolerance and an innate synchronous.
model, fault tolerance and an innate synchronous.
2
2Abstract
Abstract
 In this presentation, there are two types of Neural
 In this presentation, there are two types of Neural
Network, which are implemented for investigation.
Network, which are implemented for investigation.
Kohonen's feature map is also called as Kohonen Self-
Kohonen's feature map is also called as Kohonen Self-
O Or rg ga an niis siin ng g N Ne eu ur ra all N Ne et tw wo or rk k,, a an nd d iis s a an n u un ns su up pe er rv viis se ed d
lle ea ar rn niin ng g.. I It t h ha as s t th he e e effffiic ciie en nc cy y o off lle ea ar rn niin ng g w wiit th h n no o
guidance. Furthermore, Back Propagation Neural
guidance. Furthermore, Back Propagation Neural
Network (BP) is a type of Multilayer Perceptron in
Network (BP) is a type of Multilayer Perceptron in
FeedForward architecture and is a supervised
FeedForward architecture and is a supervised
lle ea ar rn niin ng g.. I It t h ha as s lle ea ar rn niin ng g c ca au us se ed d b by y t th he e m ma an ny y
p pe er rffo or rm ma an nc ce es s o off a a r re ec co om mm me en nd d s se et t o off t tr ra aiin niin ng g s sa am mp plle e
to Multilayer Perceptron.
to Multilayer Perceptron.
3 3
Abstract
Abstract
 A As s a a r re es su ullt t,, t th he es se e N Ne eu ur ra all N Ne et tw wo or rk ks s a ar re e a ap pp plliie ed d ffo or r
s so ollv viin ng g t th he e e ex xa am mp plle e p pr ro ob blle em ms s s su uc ch h a as s I In np pu ut t V Ve ec ct to or rs s,,
Travelling Salesman and XOR. They will also adapt for
Travelling Salesman and XOR. They will also adapt for
the research about the Biometric Thrill and Arousal
the research about the Biometric Thrill and Arousal
Detection system in the future work.
Detection system in the future work.
4
4Overview
Overview
 What is Neural Network?
 What is Neural Network?
 Why need Neural Network?
 Why need Neural Network?
 H Ho ow w t to o iim mp plle em me en nt t N Ne eu ur ra all N Ne et tw wo or rk k? ?
 N Ne eu ur ra all N Ne et tw wo or rk k E Ex xa am mp plle es s
 Conclusions and Future Work
 Conclusions and Future Work
 References
 References
5 5
What is Neural Network?
What is Neural Network?
N Ne eu ur ra all N Ne et tw wo or rk k : : t th he e r re ep pr re es se en nt ta at tiio on n o off b br ra aiin n’’s s
lle ea ar rn niin ng g a ap pp pr ro oa ac ch h
6
6What is Neural Network?
What is Neural Network?
 B Br ra aiin n : : o op pe er ra at te es s a as s m mu ullt tiip pr ro oc ce es ss so or r; ; h ha as s e ex xc ce elllle en nt t
linked
linked
 Has remarkable capability to improve the
 Has remarkable capability to improve the
problems, explain them by computers
problems, explain them by computers
 H Ha as s n ne eu ur ro on ns s,, w wh hiic ch h w wo or rk k a as s t th he e ffu un nd da am me en nt ta all
processors
processors
7 7
What is Neural Network?
What is Neural Network?
 Synapse: interconnection between neurons
 Synapse: interconnection between neurons
 I In np pu ut t: : t th he e a ac ct tiiv va at tiio on n o off a ar rr riiv viin ng g n ne eu ur ro on ns s
a ac cc cu um mu ulla at te ed d b by y t th he e w we eiig gh ht ts s o off s sy yn na ap ps se es s
 Result: the activation of neurons, which is
 Result: the activation of neurons, which is
calculated by employing a threshold function
calculated by employing a threshold function
 Parallel distributing processing; solving complicated
Parallel distributing processing; solving complicated
problems on the computers
problems on the computers
8
8What is Neural Network?
What is Neural Network?
 K Ko oh ho on ne en n
 Kohonen Self-Organizing Neural Network
 Kohonen Self-Organizing Neural Network
 U Un ns su up pe er rv viis se ed d L Le ea ar rn niin ng g: : L Le ea ar rn niin ng g w wiit th h n no o
g gu uiid da an nc ce e
 Examines only input, produce the recognition
 Examines only input, produce the recognition
of this input
of this input
9 9
What is Neural Network?
What is Neural Network?
 Kohonen
 Kohonen
 N No o h hiid dd de en n lla ay ye er rs s
 Contrast with BP: how to learn its data,
 Contrast with BP: how to learn its data,
remember a sample
remember a sample
 D Do oe es s n no ot t a ap pp plly y b bo ot th h a ac ct tiiv va at te e ffu un nc ct tiio on n a an nd d b biia as s
w we eiig gh ht t; ; c ch ho oo os se e a a w wiin nn ne er r ffr ro om m o on ne e o ou ut tp pu ut t
neurons
neurons
10
10What is Neural Network?
What is Neural Network?
 Back Propagation
 Back Propagation
 A type of Multilayer Perceptron in Feedforward
 A type of Multilayer Perceptron in Feedforward
Architecture
Architecture
 S Su up pe er rv viis se ed d L Le ea ar rn niin ng g: : L Le ea ar rn niin ng g b by y t tr ra aiin niin ng g
s sa am mp plle es s
 Learning until the synaptic weights and
Learning until the synaptic weights and
threshold are steady; the average squared
threshold are steady; the average squared
e er rr ro or r o ov ve er r t to ot ta all t tr ra aiin niin ng g s se et t iis s s sm ma alllle es st t v va allu ue e..
1 11 1
What is Neural Network?
What is Neural Network?
 B Ba ac ck k P Pr ro op pa ag ga attiio on n
 S Sa am mp plle e 3 3 lla ay ye er rs s: : iin np pu ut t,, h hiid dd de en n,, o ou ut tp pu ut t
 Given expected inputs and expected outputs;
 Given expected inputs and expected outputs;
these outputs will contrasted with predicted
these outputs will contrasted with predicted
output from BP
output from BP
 C Ca allc cu ulla at te e e er rr ro or r,, m mo od diiffy y t th he e w we eiig gh ht ts s o off lla ay ye er rs s iin n
reverse the output layer
reverse the output layer
12
12Why need Neural Network?
Why need Neural Network?
 No requirements for specifying the relevant factors
 No requirements for specifying the relevant factors
• • S St ta at tiis st tiic ca all M Mo od de ell : : n ne ee ed ds s ffa ac ct to or rs s t to o p pr ro od du uc ce e iit ts s
representation
representation
• • Neural Network : offers data and assigns data
Neural Network : offers data and assigns data
r re ella at tiio on n; ; u un nr re ella at te ed d d da at ta a h ha as s
p po oo or r c co on nn ne ec ct tiio on n s st tr re en ng gt th h
1 13 3
Why need Neural Network?
Why need Neural Network?
 A An n u un ns so op ph hiis sttiic ca atte ed d m mo od de ell
 Statistical Model : -
Statistical Model : -
 N Ne eu ur ra all N Ne et tw wo or rk k : : M Ma an ny y ffa ac ct to or rs s ffo or r p pe er rffo or rm ma an nc ce e,,
Factors help more correct results
Factors help more correct results
for the complicated problems
for the complicated problems
14
14Why need Neural Network?
Why need Neural Network?
 A Straightforward model
 A Straightforward model
 S St ta at tiis st tiic ca all M Mo od de ell : : a a r ro ou un nd da ab bo ou ut t m me et th ho od d o off lle ea ar rn niin ng g
relationship
relationship
 Neural Network : designed exactly for the problem
Neural Network : designed exactly for the problem
1 15 5
Why need Neural Network?
Why need Neural Network?
 A A F Fa au ulltt T To olle er ra an nc ce e
 Statistical Model : -
Statistical Model : -
 N Ne eu ur ra all N Ne et tw wo or rk k : : N No o p pr ro ob blle em ms s ffo or r m ma an ny y iin np pu ut t
factors, trouble in data or
factors, trouble in data or
malfunction of hardware
malfunction of hardware
16
16Why need Neural Network?
Why need Neural Network?
 An innate synchronous
 An innate synchronous
 S St ta at tiis st tiic ca all M Mo od de ell : : - -
 Neural Network : synapse can be its personalized
Neural Network : synapse can be its personalized
processor, no time dependence
processor, no time dependence
ffo or r s sy yn na ap ps se es s iin n s sa am me e lla ay ye er r a an nd d
w wo or rk k t to ot ta alllly y s sy yn nc ch hr ro on niiz za at tiio on n
1 17 7
How to implement Neural Network?
How to implement Neural Network?
 I Im mp plle em me en nt tiin ng g 2 2 K Kiin nd ds s o off N Ne eu ur ra all N Ne et tw wo or rk k
 K Ko oh ho on ne en n
 Back Propagation
Back Propagation
 P Pr ro og gr ra am mm miin ng g lla an ng gu ua ag ge e iis s C C+ ++ +
 S Su up pe er riio or r m ma an na ag ge em me en nt t iin n c clla as ss se es s
 Fast for doing the duties
Fast for doing the duties
18
18Neural Networks Examples
Neural Networks Examples
 Input Vectors problem (Kohonen)
 Input Vectors problem (Kohonen)
 T Tr ra av ve elllliin ng g S Sa alle es sm ma an n p pr ro ob blle em m ( (K Ko oh ho on ne en n) )
 X XO OR R p pr ro ob blle em m ( (B Ba ac ck k P Pr ro op pa ag ga at tiio on n) )
1 19 9
Neural Networks Examples
Neural Networks Examples
 I In np pu ut t V Ve ec ct to or rs s p pr ro ob blle em m ( (K Ko oh ho on ne en n) )
Input: 4 Input vectors
Input: 4 Input vectors
Output: 2 Weight vectors
Output: 2 Weight vectors
20
20Neural Networks Examples
Neural Networks Examples
 Input Vectors problem (Kohonen)
 Input Vectors problem (Kohonen)
2 21 1
Neural Networks Examples
Neural Networks Examples
 I In np pu ut t V Ve ec ct to or rs s p pr ro ob blle em m ( (K Ko oh ho on ne en n) )
22
22Neural Networks Examples
Neural Networks Examples
 Travelling Salesman problem (Kohonen)
 Travelling Salesman problem (Kohonen)
I In np pu ut t: : 8 8 c ciit tiie es s
O Ou ut tp pu ut t: : A An n o op pt tiim miiz ze e t tr ra av ve elllliin ng g p pa at th h ffo or r t th he es se e c ciit tiie es s
2 23 3
Neural Networks Examples
Neural Networks Examples
 T Tr ra av ve elllliin ng g S Sa alle es sm ma an n p pr ro ob blle em m ( (K Ko oh ho on ne en n) )
24
24Neural Networks Examples
Neural Networks Examples
 Travelling Salesman problem (Kohonen)
 Travelling Salesman problem (Kohonen)
2 25 5
Neural Networks Examples
Neural Networks Examples
 X XO OR R p pr ro ob blle em m ( (B Ba ac ck k P Pr ro op pa ag ga at tiio on n) )
Input: 3 input values
Input: 3 input values
Output: 1 output value
Output: 1 output value
26
26Neural Networks Examples
Neural Networks Examples
 XOR problem (Back Propagation)
 XOR problem (Back Propagation)
2 27 7
Neural Networks Examples
Neural Networks Examples
 X XO OR R p pr ro ob blle em m ( (B Ba ac ck k P Pr ro op pa ag ga at tiio on n) )
28
28Neural Networks Examples
Neural Networks Examples
 XOR problem (Back Propagation)
 XOR problem (Back Propagation)
2 29 9
Conclusion and Future Work
Conclusion and Future Work
 Neural Networks
Neural Networks
 U Un ns su up pe er rv viis se ed d L Le ea ar rn niin ng g : : K Ko oh ho on ne en n
 Supervised Learning : Back Propagation
Supervised Learning : Back Propagation
 A Ad dv va an nt ta ag ge es s a an nd d N Ne eu ur ra all N Ne et tw wo or rk k E Ex xa am mp plle es s
 A Ad da ap pt t ffo or r t th he e r re es se ea ar rc ch h a ab bo ou ut t t th he e B Biio om me et tr riic c T Th hr riillll a an nd d
A Ar ro ou us sa all D De et te ec ct tiio on n s sy ys st te em m
 Analyse, Recognize and Predict the results
 Analyse, Recognize and Predict the results
30
30Questions?
Questions?
3 31 1
References
References
 B Bllu um m,, A A 1 19 99 92 2,, N Ne eu ur ra all N Ne et tw wo or rk ks s iin n C C+ ++ +: : A An n O Ob bjje ec ct t- -
O Or riie en nt te ed d F Fr ra am me ew wo or rk k ffo or r B Bu uiilld diin ng g C Co on nn ne ec ct tiio on niis st t
Systems, John Wiley & Sons, New York
Systems, John Wiley & Sons, New York
 H He ea at to on n,, J J 2 20 00 05 5,, I In nt tr ro od du uc ct tiio on n t to o N Ne eu ur ra all N Ne et tw wo or rk ks s w wiit th h
Java (First Edition), Heaton Research, St. Louis
Java (First Edition), Heaton Research, St. Louis
 K Kw wo on ng g,, C CK K 2 20 00 01 1,, ‘‘F Fiin na an nc ciia all F Fo or re ec ca as st tiin ng g U Us siin ng g N Ne eu ur ra all
N Ne et tw wo or rk k o or r M Ma ac ch hiin ne e L Le ea ar rn niin ng g T Te ec ch hn niiq qu ue es s’’,, B Ba ac ch he ello or r
Degree Thesis, Dept. of Information Technology and
Degree Thesis, Dept. of Information Technology and
Electrical Engineering, University of Queensland
Electrical Engineering, University of Queensland
32
32References
References
 Steeb, WH, Hardy, Y & Stoop R 2008, The Nonlinear
 Steeb, WH, Hardy, Y & Stoop R 2008, The Nonlinear
workbook (Fourth Edition): chaos, fractals, cellular
workbook (Fourth Edition): chaos, fractals, cellular
a au ut to om ma at ta a,, n ne eu ur ra all n ne et tw wo or rk ks s,, g ge en ne et tiic c a allg go or riit th hm ms s,, g ge en ne e
e ex xp pr re es ss siio on n p pr ro og gr ra am mm miin ng g,, w wa av ve elle et ts s,, ffu uz zz zy y llo og giic c,, w wiit th h
C++, Java and Symbolic C++ programs, World
C++, Java and Symbolic C++ programs, World
Scientific, Singapore
Scientific, Singapore
 W We ells st te ea ad d,, S ST T 1 19 99 94 4,, N Ne eu ur ra all N Ne et tw wo or rk k a an nd d F Fu uz zz zy y L Lo og giic c
Applications in C/C++, John Wiley & Sons, New York
Applications in C/C++, John Wiley & Sons, New York
3 33 3