Neural Networks

glibdoadingΤεχνίτη Νοημοσύνη και Ρομποτική

20 Οκτ 2013 (πριν από 3 χρόνια και 9 μήνες)

85 εμφανίσεις

Neural Networks

NN 1

1

Neural Networks

Teacher:

Elena Marchiori

R4.47

elena@cs.vu.nl

Assistant:


Kees Jong

S2.22

cjong@cs.vu.nl


Neural Networks

NN 1

2

Course Outline

Basics of neural network theory and practice for
supervised

and
unsupervised

learning.


Most popular
Neural Network models
:


architectures


learning algorithms


applications

Neural Networks

NN 1

3

Course Outline

Rules:

-

4 s.p


-

Final mark is based on
two

assignments
, which will
be available at the end of the course.


-

one assignment is on
theory

(to do
alone
).


-

one assignment is on
practice

(to do in
couples).


-

Programming in
Matlab

5.3.


-

Registration: send
email

to
cjong@cs.vu.nl

Neural Networks

NN 1

4

Course Organization



There is no text book.



Course schedule, slides and exercises will be
available at



http://www.cs.vu.nl/~elena/nn.html


Neural Networks

NN 1

5

Neural Networks


A NN is a machine learning approach inspired by the
way in which the brain performs a particular learning
task
:


Knowledge about the learning task is given in the form of
examples
.



Inter neuron connection strengths (
weights
) are used to
store

the acquired

information (the training examples).



During the
learning process

the weights are modified in
order to model the particular learning task correctly on the
training examples
.

Neural Networks

NN 1

6


Supervised Learning


Recognizing hand
-
written digits,

pattern recognition,
regression.


Labeled examples






(input , desired output)


Neural Network models:
perceptron
,
feed
-
forward
,
radial
basis function
,
support vector machine
.


Unsupervised Learning


Find similar groups of documents in the web,

content
addressable memory, clustering.


Unlabeled examples






(different realizations of the input alone)


Neural Network models:
self organizing maps
,
Hopfield
networks
.


Learning

Neural Networks

NN 1

7

Network architectures


Three different classes of network architectures



single
-
layer feed
-
forward

neurons are organized


multi
-
layer feed
-
forward

in acyclic layers


recurrent




The

architecture

of a neural network is linked with the
learning algorithm used to train



Neural Networks

NN 1

8

Single Layer Feed
-
forward

Input layer

of

source nodes

Output layer

of

neurons

Neural Networks

NN 1

9

Multi layer feed
-
forward

Input

layer

Output

layer

Hidden Layer

3
-
4
-
2 Network

Neural Networks

NN 1

10

Recurrent Network with
hidden neuron(s)
: unit
delay operator
z
-
1
implies dynamic system


z
-
1

z
-
1

z
-
1


Recurrent network

input

hidden

output

Neural Networks

NN 1

11

Neural Network Architectures

Neural Networks

NN 1

12

The Neuron


The neuron is the basic information processing unit of
a NN. It consists of:

1
A set of
synapses

or
connecting links
, each link
characterized by a
weight
:


W
1
, W
2
, …, W
m

2
An

adder

function (linear combiner) which
computes the weighted sum of
the inputs:



3
Activation function

(squashing function) for
limiting the amplitude of the
output of the neuron.

Neural Networks

NN 1

13

The Neuron

Input

signal

Synaptic

weights

Summing

function

Bias

b

Activation

function

Local

Field

v

Output

y

x
1

x
2

x
m

w
2

w
m

w
1

Neural Networks

NN 1

14

Bias of a Neuron


Bias
b

has the effect of applying an
affine
transformation

to
u

v = u + b


v

is the
induced field

of the neuron

v

u

Neural Networks

NN 1

15

Bias as extra input

Input

signal

Synaptic

weights

Summing

function

Activation

function

Local

Field

v

Output

y

x
1

x
2

x
m

w
2

w
m

w
1

w
0

x
0
= +1



Bias is an external parameter of the neuron
. C
an be
modeled by adding an extra input.

Neural Networks

NN 1

16

Dimensions of a Neural
Network



Various types of
neurons



Various network
architectures



Various
learning algorithms



Various
applications

Neural Networks

NN 1

17

Face Recognition

90% accurate learning head pose, and recognizing 1
-
of
-
20 faces

Neural Networks

NN 1

18

Handwritten digit recognition