CSCI

495
Artificial Intelligence
Lecture
29
Neural Networks
Biology
•
The brain uses massively parallel computation
–
10
11
neurons in the brain
–
10
4
connections per neuron
Slide Adapted from
:
M. Hagan, H. Demuth, M. Beale,
Neural Network Design
Single

Input Neuron
Slide Adapted from
:
M. Hagan, H. Demuth, M. Beale,
Neural Network Design
Single

Input Neuron
•
w
= 3,
p
= 2,
b
=

1.5, what is
a
?
•
Note that both
w
and
b
are adjustable scalar parameters
of the neuron. Typically the transfer function is chosen
by the designer and the parameters are adjusted by
some learning rule so that the input/output of the neuron
meets some specific goal
Slide Adapted from
:
M. Hagan, H. Demuth, M. Beale,
Neural Network Design
Multiple

Input Neuron
Slide Adapted from
:
M. Hagan, H. Demuth, M. Beale,
Neural Network Design
Abbreviated Notation
Slide Adapted from
:
M. Hagan, H. Demuth, M. Beale,
Neural Network Design
Linear Classifiers
•
Linear classifier
single linear decision boundary
(for 2

class case)
•
We can always represent a linear decision boundary by a
linear equation:
w
1
p
1
+
w
1
p
2
+ … +
w
1
p
R
+
b
=
S
w
i
p
i
+
b
=
Wp
+
b
= 0
•
In R dimensions, this defines a (R

1) dimensional
hyperplane
–
R=3, we get a plane; R=2, we get a line
•
For prediction we simply see if
S
w
i
p
i
+
b
> 0
•
The
w
i
are the weights (parameters)
–
Learning consists of searching in the R

dimensional weight space
for the set of weights (the linear boundary) that minimizes an error
measure
(DIAGRAM DONE IN CLASS)
Perceptron
Slide Adapted from
:
M. Hagan, H. Demuth, M. Beale,
Neural Network Design
Two

Input Case First
Slide Adapted from
:
M. Hagan, H. Demuth, M. Beale,
Neural Network Design
Decision Boundary
Apple/Banana Sorter
Slide Adapted from
:
M. Hagan, H. Demuth, M. Beale,
Neural Network Design
Apple/Banana Sorter
Slide Adapted from
:
M. Hagan, H. Demuth, M. Beale,
Neural Network Design
Prototype Banana
Prototype Apple
Shape
: {1 : round ;

1 : elliptical}
Texture
: {1 : smooth ;

1 : rough}
Weight
: {1 : > 1 lb. ;

1 : < 1 lb.}
Measurement Vector
The decision boundary should separate the prototype vectors
The
weight
vector
should
be
orthogonal
to
the
decision
boundary,
and
should
point
in
the
direction
of
the
vector
which
should
produce
an
output
of
1
.
The
bias
determines
the
position
of
the
boundary
(
DONE
IN
CLASS
)
Testing the Network
Slide Adapted from
:
M. Hagan, H. Demuth, M. Beale,
Neural Network Design
Banana:
Apple:
“Rough” Banana:
Comments 0
Log in to post a comment