Neural Network
Ming

Feng Yeh (
葉明豐
)
Department of Electrical Engineering
Lunghwa University of Science and Technology
E

mail: mfyeh@mail.lhu.edu.tw
Office: F412

III Tel: #5518
Ming

Feng Yeh
2
COURSE OBJECTIVE
This course gives an introduction to
basic
neural network architectures
and
learning
rules
.
Emphasis is placed on the mathematical
analysis of these networks, on methods of
training them and on their application to
practical engineering problems
in such
areas as
pattern recognition
,
signal
processing
and
control systems
.
Ming

Feng Yeh
3
SYLLABUS
Textbook
:
Hagan, Demuth, Beale
,
Neural Network Design
,
PWS Publishing Company
Midterm Exam
: 30%
Final Exam
: 30%
Projects:
40%
Ming

Feng Yeh
4
CONTENTS
Ch 1.
Introduction
Ch 2.
Neuron Model & Neural Architecture
Ch 3&4.
Perceptron
(
感知機
)
Learning Rule
Ch 7.
Supervised
(
監督式
)
Hebbian Learning
Ch 10.
Widrow

Hoff
Learning
Ch 11&12.
Back

propagation
(
倒傳遞
)
Ch 13.
Associative
(
關聯
)
Learning
Ch 14.
Competitive
(
競爭
)
Networks
Ch 15.
Grossberg
Networks
Ch 16.
Adaptive Resonance
(
自適應
)
Theory
Ch 18.
Hopfield
Network
Ming

Feng Yeh
5
Information
Review
Ch 5
–
Signal and Weight Vector Spaces
Ch 6
–
Linear Transformations for
Neural Networks
Ming

Feng Yeh
6
CHAPTER 1
Introduction
Ming

Feng Yeh
7
Objectives
As you read these words you are using a
complex
biological neural network
. You
have a highly interconnected set of
10
11
neurons
to facilitate your
reading
,
breathing
,
motion
and
thinking
.
In the
artificial neural network
, the neurons
are
not
biological. They are extremely simple
abstractions of biological neurons, realized as
elements
in a program or perhaps as
circuits
made of silicon.
Ming

Feng Yeh
8
History

1
Pre

1940: von Hemholtz, Mach & Pavlov
General theories of learning, vision, conditioning
No specific mathematical models of neuron operation
1940s: Hebb, McCulloch & Pitts
Mechanism for learning in biological neurons (Hebb)
Neural

like networks
can compute any arithmetic or logical
function (McCulloch & Pitts)
1950s: Rosenblatt, Widrow & Hoff
First practical networks and learning rules:
the perception
network
and associated learning rule (Rosenblatt) &
Widrow

Hoff learning rule
Can not successfully modify their learning rules to train the
more complex networks.
Ming

Feng Yeh
9
History

2
1960s: Minsky & Papert
Demonstrated limitations of existing neural networks
Neural network research was largely suspended
1970s: Kohonen, Anderson & Grossberg
Kohonen and Anderson independently and separately
developed neural networks that could as
memories
Self

organizing networks
(Grossberg)
1980s: Hopfield, Rumelhart & McClelland
The use of statistical mechanics to explain the operation of
recurrent network: an
associative memory
(Hopfield)
Backpropagation algorithm
(Rumelhart & McClelland)
Ming

Feng Yeh
10
Applications
The applications are expanding
because neural networks are good at
solving problems, not just in
engineering
,
science
and
mathematics
, but in
medicine
,
business
,
finance
and
literature
as
well.
Ming

Feng Yeh
11
Biological Inspiration
Human brain
consists of a large number
(about
10
11
) of highly interconnected
elements (about
10
4
connections per
element) called
neurons (
神經元
)
.
Three principle components are the
dendrites
, the
cell body
and the
axon
.
The point of contact is called a
synapse
.
Ming

Feng Yeh
12
Biological Neurons
Dendrites
(
樹突
⤺)
carry
electrical
into
the cell body
Dendrites
Cell Body
Cell Body
(
細胞體
⤺)
sums
and
thresholds
these incoming
signals
Axon
(
軸突
⤺)
carry
the signal
from the cell body
out
to other
neurons
Axon
Synapse
(
突觸
⤺
contact
between an axon of one cell
and a dendrites of another cell
Synapse
Soma
Ming

Feng Yeh
13
Neural Networks
Neural Networks
: a promising new
generation of
information processing systems
,
usually operate in parallel
, that demonstrate
the ability to
learn
,
recall
, and
generalize
from training patterns or data.
Basic models
,
learning rules
, and
distributed
representations
of neural networks will be
discussed.
Ming

Feng Yeh
14
補充資料
Artificial neural network
可譯為
類神經網路
或
人
工神經網路
，是指
模仿
生物神經網路的一種
資
訊處理系統
。
類神經網路是一種
計算系統
，包括
軟體
與
硬體
，
它使用大量簡單的相連人工神經元來模仿生物
神經網路的能力。
人工神經元是生物神經元的
簡單模擬
，它從
外界環境
或
其它人工神經元
取
得資訊，並加以簡單的
運算
，並輸出其結果到
外界環境或其它人工神經元。
Ming

Feng Yeh
15
Fuzzy Logic
Fuzzy set theory
was first proposed by
Lotfi Zadeh in 1965.
A mathematical way to represent
vagueness
in
linguistics
A generalization of
classical set theory
Ming

Feng Yeh
16
Fuzzy Systems v.s.
Neural Networks
Fuzzy logic
is based on the way the
brain
deals with
inexact information
.
Neural networks
are modeled after the
physical architecture
of the
brain
.
Fuzzy systems and neural networks are both
numerical model

free estimator
and
dynamical systems
.
They share the common ability to improve the
intelligence
of systems working in an
uncertain
,
imprecise
and
noisy environment
.
Ming

Feng Yeh
17
Machine Intelligence
Neural networks
provide fuzzy
systems with
learning ability
.
Fuzzy systems
provide neural
networks with a structure framework
with high

level
fuzzy
IF

THEN rule
thinking
and
reasoning
.
Ming

Feng Yeh
18
Fuzzy Neural
Integrated System
Neural fuzzy systems
: use of neural
networks as tools in
fuzzy models
.
Fuzzy neural networks
: fuzzification of
conventional neural
network models
.
Fuzzy

neural hybrid systems
:
incorporation of fuzzy logic technology
and neural networks into
hybrid
systems
.
Ming

Feng Yeh
19
Soft
/
Hard Computing
Hard computing
whose prime desiderata are
precision
,
certainty
, and
rigor
.
Soft computing
is tolerant of
imprecision
,
uncertainty
, and
partial truth
.
(Lotfi Zadeh)
The primary aim of
soft computing
is to exploit
such tolerance to achieve
tractability
,
robustness
,
a
high level
of
machine intelligence
, and a
low
cost
in practical applications.
Fuzzy logic
,
neural networks
(including
CMAC
),
probabilistic reasoning
(
genetic
algorithm,
evolutionary
programming, and
chaotic
systems)
Ming

Feng Yeh
20
Soft Computing
Methodology
Strength
Neural network
Learning and adaptation
Fuzzy set theory
Knowledge representation
via fuzzy if

then rule
Genetic algorithm
and simulated
annealing
Systematic random search
Conventional AI
Symbolic manipulation
Ming

Feng Yeh
21
Computational Intelligence
Fuzzy logic
,
neural network
,
genetic algorithm
,
and
evolutionary programming
are also
considered the building blocks of
computational intelligence
.
(James Bezdek)
Computational intelligence is
low

level
cognition
in the style of human brain and is
contrast to conventional (symbolic)
artificial
intelligence (AI)
.
Comments 0
Log in to post a comment