neural network

muscleblouseAI and Robotics

Oct 19, 2013 (4 years and 4 months ago)

97 views

Neural

Network

By: Lv Pengfei

Contents:


Introduction to ANN


Development History


Classification


Capabilities



Shortages


Application Areas

2

Date: 09
-
04
-
2013 E
-
mail: Pengfei.lu@uta.fi

Introduction:


Biological neural network(BNN)


Artifical neural
network(ANN)





An
artificial neural network
, often just named a
neural network
, is a
mathematical model

inspired by
biological neural networks
.


-
Wikipedia

3

Date: 09
-
04
-
2013 E
-
mail: Pengfei.lu@uta.fi

BNN(1/2):

4

Date: 09
-
04
-
2013 E
-
mail: Pengfei.lu@uta.fi


10
billion neurons in human brain


Summation of input stimuli


Spatial (signals)


Temporal (pulses
)


Constant firing strength




billion synapses in human brain


Chemical transmission and modulation
of signals


Inhibitory synapses


Excitatory synapses

BNN(2/2):

5

Date: 09
-
04
-
2013 E
-
mail: Pengfei.lu@uta.fi


10,
000
~100,000

synapses
per neuron


Computational power =
connectivity


Plasticity


new connections (?)


strength of connections
modified

ANN
-

Concepts(1/7):

6

Date: 09
-
04
-
2013 E
-
mail: Pengfei.lu@uta.fi



A
neural network consists of an interconnected group of
artificial neurons
, and it
processes information

using a
connectionist approach to
computation
.



In
most cases a neural network is an
adaptive system

changing its structure during a
learning phase
.



Neural
networks are used for

modeling complex
relationships between inputs and outputs or to find
patterns in data
.



-
Wikipedia

ANN
-

Hebb’s
Postulate of
Learning
(2/7):

7

Date: 09
-
04
-
2013 E
-
mail: Pengfei.lu@uta.fi


When
an axon of cell A is near enough to excite a cell
and repeatedly or persistently takes part in firing it,
some growth process or metabolic change takes
place in one or both cells such that A’s efficiency as
one of the cells firing B is increased.


ANN
-

Hebb’s Postulate: revisited
(3/7):

8

Date: 09
-
04
-
2013 E
-
mail: Pengfei.lu@uta.fi


Stent (1973),

Changeux

and
Danchin

(1976
)


Inhibitory synapses


Synchronously
-
> increased


Asynchronously
-
>
weakened



ANN
-

Artificial
Neuron(4/7):

9

Date: 09
-
04
-
2013 E
-
mail: Pengfei.lu@uta.fi

ANN
-

Layers(5/7):

10

Date: 09
-
04
-
2013 E
-
mail: Pengfei.lu@uta.fi

Multilayer Feedforward
Network



Input layer

Hidden layer

Output layer

ANN
-

Example(6/7):

11

Date: 09
-
04
-
2013 E
-
mail: Pengfei.lu@uta.fi

Handwriting
recognition


program

1

0

B

A

ANN
-

Example(7/7):

12

Date: 09
-
04
-
2013 E
-
mail: Pengfei.lu@uta.fi





















Contents:


Introduction to ANN


Development History


Classification


Capabilities



Shortages


Application Areas

13

Date: 09
-
04
-
2013 E
-
mail: Pengfei.lu@uta.fi

Development
History(1/2)

14

Date: 09
-
04
-
2013 E
-
mail: Pengfei.lu@uta.fi


Early stages


1943 McCulloch
-
Pitts: neuron as comp. elem.


1948 Wiener: cybernatics


1949 Hebb: learning rule


1958 Rosenblatt: perceptron


1960 Widrow
-
Hoff: least mean square
algorithm

Development
History(2/2)

15

Date: 09
-
04
-
2013 E
-
mail: Pengfei.lu@uta.fi


Recession


1969 Minsky
-
Papert: limitations perceptron model


Revival


1982 Hopfield: recurrent network model


1982 Kohonen: self
-
organizing maps


1986
Rumelhart, Hinton, Willliams.
al.: backpropagation


1990~1999
Decade
of brain


Contents:


Introduction to ANN


History of development


Classification



Capabilities



Shortages


Application Areas

16

Date: 09
-
04
-
2013 E
-
mail: Pengfei.lu@uta.fi

Classification
-

Algorithm
:

17

Date: 09
-
04
-
2013 E
-
mail: Pengfei.lu@uta.fi


Supervised
Learning
Network*


Unsupervised
Learning
Network


Hybrid
Learning
Network


Associate
Learning
Network


Optimization
Application
Network

Classification
-

Connectionism
:

18

Date: 09
-
04
-
2013 E
-
mail: Pengfei.lu@uta.fi


Feed
Forward
Network


Recurrent Network


Reinforcement Network

Contents:


Introduction to ANN


History of development


Classification


Capabilities



Shortages


Application Areas

19

Date: 09
-
04
-
2013 E
-
mail: Pengfei.lu@uta.fi

Capabilities
:

20

Date: 09
-
04
-
2013 E
-
mail: Pengfei.lu@uta.fi


Learning


Approximate reasoning


Generalisation capability


Noise
filtering


Parallel processing


Distributed knowledge
base


Fault tolerance

Contents:


Introduction to ANN


History of development


Classification


Capabilities



Shortages



Application Areas

21

Date: 09
-
04
-
2013 E
-
mail: Pengfei.lu@uta.fi

S
hortages:

22

Date: 09
-
04
-
2013 E
-
mail: Pengfei.lu@uta.fi


Knowledge base not transparent (black
box) (Partially resolved)


Learning sometimes difficult/slow


Limited storage capability

Contents:


Introduction to ANN


History of development


Classification


Capabilities



Shortages


Application Areas

23

Date: 09
-
04
-
2013 E
-
mail: Pengfei.lu@uta.fi

Application
Areas:

24

Date: 09
-
04
-
2013 E
-
mail: Pengfei.lu@uta.fi


Classification


Clustering


Associative memory


Control


Function approximation

ANN Clustering Applications
:

25

Date: 09
-
04
-
2013 E
-
mail: Pengfei.lu@uta.fi


Natural language processing


Document clustering


Document retrieval


Automatic query


Image segmentation


Data mining


Data set partitioning


Detection of emerging clusters


Fuzzy partitioning


Condition
-
action association

Face recognition

Date: 09
-
04
-
2013 E
-
mail: Pengfei.lu@uta.fi

26

Application of ANN Associative Memories
:

27

Date: 09
-
04
-
2013 E
-
mail: Pengfei.lu@uta.fi


Character recognition


Handwriting recognition


Noise filtering


Data compression


Information retrieval

Handwriting Characters

Recognition

Date: 09
-
04
-
2013 E
-
mail: Pengfei.lu@uta.fi

28

ANN Control Applications
:

29

Date: 09
-
04
-
2013 E
-
mail: Pengfei.lu@uta.fi


Non
-
linear process control


Chemical reaction control


Industrial process control


Water treatment


Intensive care of patients


Dynamic
system control


Helicopter flight control


Underwater robot control


Servo control


Robot manipulators


Autonomous vehicles


Automotive control

ANN Modelling
Applications
:

30

Date: 09
-
04
-
2013 E
-
mail: Pengfei.lu@uta.fi


Modelling of highly nonlinear industrial
processes


Financial market prediction


Weather forecasts


River flow prediction


Fault/breakage prediction


Monitoring of critically ill patients

Thanks

Questions?