Neural Network

appliancepartΤεχνίτη Νοημοσύνη και Ρομποτική

19 Οκτ 2013 (πριν από 3 χρόνια και 7 μήνες)

53 εμφανίσεις

Neural Network

Ming
-
Feng Yeh (
葉明豐
)

Department of Electrical Engineering

Lunghwa University of Science and Technology

E
-
mail: mfyeh@mail.lhu.edu.tw

Office: F412
-
III Tel: #5518

Ming
-
Feng Yeh

2

COURSE OBJECTIVE

This course gives an introduction to
basic
neural network architectures

and
learning
rules
.

Emphasis is placed on the mathematical
analysis of these networks, on methods of
training them and on their application to
practical engineering problems

in such
areas as
pattern recognition
,
signal
processing

and
control systems
.

Ming
-
Feng Yeh

3

SYLLABUS

Textbook
:
Hagan, Demuth, Beale
,

Neural Network Design
,

PWS Publishing Company

Midterm Exam
: 30%

Final Exam
: 30%

Projects:

40%

Ming
-
Feng Yeh

4

CONTENTS

Ch 1.

Introduction

Ch 2.

Neuron Model & Neural Architecture

Ch 3&4.

Perceptron
(
感知機
)

Learning Rule

Ch 7.

Supervised
(
監督式
)

Hebbian Learning

Ch 10.

Widrow
-
Hoff

Learning

Ch 11&12.

Back
-
propagation
(
倒傳遞
)


Ch 13.

Associative
(
關聯
)

Learning

Ch 14.

Competitive
(
競爭
)

Networks

Ch 15.

Grossberg

Networks

Ch 16.

Adaptive Resonance
(
自適應
)

Theory

Ch 18.

Hopfield

Network

Ming
-
Feng Yeh

5

Information

Review


Ch 5



Signal and Weight Vector Spaces



Ch 6



Linear Transformations for


Neural Networks

Ming
-
Feng Yeh

6

CHAPTER 1

Introduction

Ming
-
Feng Yeh

7

Objectives

As you read these words you are using a
complex
biological neural network
. You
have a highly interconnected set of
10
11

neurons

to facilitate your
reading
,
breathing
,
motion

and
thinking
.

In the
artificial neural network
, the neurons
are
not

biological. They are extremely simple
abstractions of biological neurons, realized as
elements

in a program or perhaps as
circuits

made of silicon.

Ming
-
Feng Yeh

8

History
-
1

Pre
-
1940: von Hemholtz, Mach & Pavlov


General theories of learning, vision, conditioning


No specific mathematical models of neuron operation

1940s: Hebb, McCulloch & Pitts


Mechanism for learning in biological neurons (Hebb)


Neural
-
like networks

can compute any arithmetic or logical
function (McCulloch & Pitts)

1950s: Rosenblatt, Widrow & Hoff


First practical networks and learning rules:
the perception
network

and associated learning rule (Rosenblatt) &
Widrow
-
Hoff learning rule


Can not successfully modify their learning rules to train the
more complex networks.

Ming
-
Feng Yeh

9

History
-
2

1960s: Minsky & Papert


Demonstrated limitations of existing neural networks


Neural network research was largely suspended

1970s: Kohonen, Anderson & Grossberg


Kohonen and Anderson independently and separately
developed neural networks that could as
memories


Self
-
organizing networks

(Grossberg)

1980s: Hopfield, Rumelhart & McClelland


The use of statistical mechanics to explain the operation of
recurrent network: an
associative memory

(Hopfield)


Backpropagation algorithm

(Rumelhart & McClelland)

Ming
-
Feng Yeh

10

Applications

The applications are expanding
because neural networks are good at
solving problems, not just in
engineering
,
science

and
mathematics
, but in
medicine
,
business
,
finance

and
literature
as
well.

Ming
-
Feng Yeh

11

Biological Inspiration

Human brain

consists of a large number
(about
10
11
) of highly interconnected
elements (about
10
4

connections per
element) called
neurons (
神經元
)
.

Three principle components are the
dendrites
, the
cell body

and the
axon
.

The point of contact is called a
synapse
.

Ming
-
Feng Yeh

12

Biological Neurons

Dendrites
(
樹突
⤺)
carry

electrical
into

the cell body

Dendrites

Cell Body

Cell Body
(
細胞體
⤺)
sums

and
thresholds

these incoming
signals

Axon
(
軸突
⤺)
carry

the signal
from the cell body
out

to other
neurons

Axon

Synapse
(
突觸


contact

between an axon of one cell
and a dendrites of another cell

Synapse

Soma

Ming
-
Feng Yeh

13

Neural Networks

Neural Networks
: a promising new
generation of
information processing systems
,
usually operate in parallel
, that demonstrate
the ability to
learn
,
recall
, and
generalize

from training patterns or data.

Basic models
,
learning rules
, and
distributed
representations

of neural networks will be
discussed.

Ming
-
Feng Yeh

14

補充資料

Artificial neural network
可譯為
類神經網路


工神經網路
,是指
模仿
生物神經網路的一種

訊處理系統


類神經網路是一種
計算系統
,包括
軟體

硬體

它使用大量簡單的相連人工神經元來模仿生物
神經網路的能力。
人工神經元是生物神經元的
簡單模擬
,它從
外界環境

其它人工神經元

得資訊,並加以簡單的
運算
,並輸出其結果到
外界環境或其它人工神經元。

Ming
-
Feng Yeh

15

Fuzzy Logic

Fuzzy set theory

was first proposed by
Lotfi Zadeh in 1965.

A mathematical way to represent
vagueness

in
linguistics

A generalization of
classical set theory

Ming
-
Feng Yeh

16

Fuzzy Systems v.s.
Neural Networks

Fuzzy logic

is based on the way the
brain

deals with
inexact information
.

Neural networks

are modeled after the
physical architecture

of the
brain
.

Fuzzy systems and neural networks are both
numerical model
-
free estimator

and
dynamical systems
.

They share the common ability to improve the
intelligence

of systems working in an
uncertain
,
imprecise

and
noisy environment
.

Ming
-
Feng Yeh

17

Machine Intelligence

Neural networks

provide fuzzy
systems with
learning ability
.

Fuzzy systems

provide neural
networks with a structure framework
with high
-
level
fuzzy

IF
-
THEN rule

thinking

and
reasoning
.

Ming
-
Feng Yeh

18

Fuzzy Neural

Integrated System

Neural fuzzy systems
: use of neural
networks as tools in
fuzzy models
.

Fuzzy neural networks
: fuzzification of
conventional neural
network models
.

Fuzzy
-
neural hybrid systems
:
incorporation of fuzzy logic technology
and neural networks into
hybrid
systems
.


Ming
-
Feng Yeh

19

Soft
/
Hard Computing

Hard computing

whose prime desiderata are
precision
,
certainty
, and
rigor
.

Soft computing

is tolerant of
imprecision
,
uncertainty
, and
partial truth
.
(Lotfi Zadeh)

The primary aim of
soft computing

is to exploit
such tolerance to achieve
tractability
,
robustness
,
a
high level

of
machine intelligence
, and a
low
cost

in practical applications.

Fuzzy logic
,
neural networks
(including
CMAC
),
probabilistic reasoning

(
genetic

algorithm,
evolutionary

programming, and
chaotic

systems)

Ming
-
Feng Yeh

20

Soft Computing

Methodology

Strength

Neural network

Learning and adaptation

Fuzzy set theory

Knowledge representation
via fuzzy if
-
then rule

Genetic algorithm
and simulated
annealing

Systematic random search

Conventional AI

Symbolic manipulation

Ming
-
Feng Yeh

21

Computational Intelligence

Fuzzy logic
,
neural network
,
genetic algorithm
,
and
evolutionary programming

are also
considered the building blocks of
computational intelligence
.
(James Bezdek)

Computational intelligence is
low
-
level
cognition

in the style of human brain and is
contrast to conventional (symbolic)
artificial
intelligence (AI)
.