Neural Networks
NN 1
1
Neural Networks
Teacher:
Elena Marchiori
R4.47
elena@cs.vu.nl
Assistant:
Kees Jong
S2.22
cjong@cs.vu.nl
Neural Networks
NN 1
2
Course Outline
Basics of neural network theory and practice for
supervised
and
unsupervised
learning.
Most popular
Neural Network models
:
•
architectures
•
learning algorithms
•
applications
Neural Networks
NN 1
3
Course Outline
Rules:

4 s.p

Final mark is based on
two
assignments
, which will
be available at the end of the course.

one assignment is on
theory
(to do
alone
).

one assignment is on
practice
(to do in
couples).

Programming in
Matlab
5.3.

Registration: send
email
to
cjong@cs.vu.nl
Neural Networks
NN 1
4
Course Organization
•
There is no text book.
•
Course schedule, slides and exercises will be
available at
http://www.cs.vu.nl/~elena/nn.html
Neural Networks
NN 1
5
Neural Networks
•
A NN is a machine learning approach inspired by the
way in which the brain performs a particular learning
task
:
–
Knowledge about the learning task is given in the form of
examples
.
–
Inter neuron connection strengths (
weights
) are used to
store
the acquired
information (the training examples).
–
During the
learning process
the weights are modified in
order to model the particular learning task correctly on the
training examples
.
Neural Networks
NN 1
6
•
Supervised Learning
–
Recognizing hand

written digits,
pattern recognition,
regression.
–
Labeled examples
(input , desired output)
–
Neural Network models:
perceptron
,
feed

forward
,
radial
basis function
,
support vector machine
.
•
Unsupervised Learning
–
Find similar groups of documents in the web,
content
addressable memory, clustering.
–
Unlabeled examples
(different realizations of the input alone)
–
Neural Network models:
self organizing maps
,
Hopfield
networks
.
Learning
Neural Networks
NN 1
7
Network architectures
•
Three different classes of network architectures
–
single

layer feed

forward
neurons are organized
–
multi

layer feed

forward
in acyclic layers
–
recurrent
•
The
architecture
of a neural network is linked with the
learning algorithm used to train
Neural Networks
NN 1
8
Single Layer Feed

forward
Input layer
of
source nodes
Output layer
of
neurons
Neural Networks
NN 1
9
Multi layer feed

forward
Input
layer
Output
layer
Hidden Layer
3

4

2 Network
Neural Networks
NN 1
10
Recurrent Network with
hidden neuron(s)
: unit
delay operator
z

1
implies dynamic system
z

1
z

1
z

1
Recurrent network
input
hidden
output
Neural Networks
NN 1
11
Neural Network Architectures
Neural Networks
NN 1
12
The Neuron
•
The neuron is the basic information processing unit of
a NN. It consists of:
1
A set of
synapses
or
connecting links
, each link
characterized by a
weight
:
W
1
, W
2
, …, W
m
2
An
adder
function (linear combiner) which
computes the weighted sum of
the inputs:
3
Activation function
(squashing function) for
limiting the amplitude of the
output of the neuron.
Neural Networks
NN 1
13
The Neuron
Input
signal
Synaptic
weights
Summing
function
Bias
b
Activation
function
Local
Field
v
Output
y
x
1
x
2
x
m
w
2
w
m
w
1
Neural Networks
NN 1
14
Bias of a Neuron
•
Bias
b
has the effect of applying an
affine
transformation
to
u
v = u + b
•
v
is the
induced field
of the neuron
v
u
Neural Networks
NN 1
15
Bias as extra input
Input
signal
Synaptic
weights
Summing
function
Activation
function
Local
Field
v
Output
y
x
1
x
2
x
m
w
2
w
m
w
1
w
0
x
0
= +1
•
Bias is an external parameter of the neuron
. C
an be
modeled by adding an extra input.
Neural Networks
NN 1
16
Dimensions of a Neural
Network
•
Various types of
neurons
•
Various network
architectures
•
Various
learning algorithms
•
Various
applications
Neural Networks
NN 1
17
Face Recognition
90% accurate learning head pose, and recognizing 1

of

20 faces
Neural Networks
NN 1
18
Handwritten digit recognition
Comments 0
Log in to post a comment