Neural Network

unknownlippsΤεχνίτη Νοημοσύνη και Ρομποτική

16 Οκτ 2013 (πριν από 4 χρόνια και 27 μέρες)

89 εμφανίσεις

Computer Science Department FMIPA IPB 2003

Neural Computing

Yeni Herdiyeni

Computer Science Dept. FMIPA IPB

Computer Science Department FMIPA IPB 2003

Neural Computing : The Basic


Artificial Neural Networks (ANN)



Mimics How Our Brain Works



Machine Learning


Neural Computing = Artificial Neural Networks (ANNs)

Computer Science Department FMIPA IPB 2003

Machine Learning : Overview


ANN to automate complex decision making



Neural networks learn from past experience
and improve their performance levels



Machine learning:
methods that teach
machines to solve problems or to support
problem solving, by applying historical cases


Computer Science Department FMIPA IPB 2003

Neural Network and Expert System

Different technologies
complement

each
other


Expert systems: logical, symbolic
approach


Neural networks: model
-
based, numeric
and associative processing


Computer Science Department FMIPA IPB 2003

Expert System


Good for closed
-
system applications
(literal and precise inputs, logical
outputs)


Reason with established facts and pre
-
established rules


Computer Science Department FMIPA IPB 2003

Major Limitation ES


Experts do not always think in terms of rules



Experts may not be able to explain their line
of reasoning



Experts may explain incorrectly



Sometimes difficult or impossible to build
knowledge base


Computer Science Department FMIPA IPB 2003

Neural Computing Use :

Neural Networks in Knowledge Acquisition



Fast identification of implicit knowledge by
automatically analyzing cases of historical data



ANN identifies
patterns

and relationships that may
lead to rules for expert systems



A trained neural network can rapidly process
information to produce associated facts and
consequences


Computer Science Department FMIPA IPB 2003

Benefit NN


Pattern recognition, learning, classification,
generalization and abstraction, and interpretation of
incomplete and noisy inputs


Character, speech and visual recognition


Can provide some human problem
-
solving
characteristics


Can tackle new kinds of problems


Robust


Fast


Flexible and easy to maintain


Powerful hybrid systems


Computer Science Department FMIPA IPB 2003

Biology Analogy : Biological Neural Network


Neurons: brain cells


Nucleus (at the center)


Dendrites provide inputs


Axons send outputs


Synapses increase or decrease
connection strength and cause
excitation or inhibition of subsequent
neurons

Computer Science Department FMIPA IPB 2003

Biology Analogy : Biological Neural Network

Computer Science Department FMIPA IPB 2003

Neural Network ?


Neural Network is a networks of many
simple processors, each possibly
having a small amount of local
memory.


The processors are connected with
communication channels (synapses).



Computer Science Department FMIPA IPB 2003

Neural Network (Haykin*)


Neural Network is a massively parallel
-
distributed processor that
has a
natural prosperity for storing
experiential knowledge and making it
available for use.



Simon Haykin, Neural Networks: A Comprehensive Foundation, Prentice Hall
Inc., New Jersey, 1999.

Computer Science Department FMIPA IPB 2003

Neural Net = Brain ?

1.
Knowledge is acquired by the
network through a learning process.

2.
Inter
-
neuron connection strengths
known as synaptic weights are used
to store the knowledge.



Computer Science Department FMIPA IPB 2003

Neural Network Fundamentals


Components and Structure


Processing Elements


Network


Structure of the Network


Processing Information by the Network


Inputs


Outputs


Weights


Summation Function


Computer Science Department FMIPA IPB 2003

Processing Information in
an Artificial Neuron

x
1

w
1j

x
2

x
i

Y
j

w
ij

w
2j

Neuron j



w
ij
x
i

Weights

Output

Inputs

Summations

Transfer function



Computer Science Department FMIPA IPB 2003

Learning : 3 Tasks


1. Compute Outputs


2. Compare Outputs with Desired Targets


3. Adjust Weights and Repeat the Process



Computer Science Department FMIPA IPB 2003

Training The Network


Present the
training data

set to the
network


Adjust weights

to produce the desired
output for each of the inputs


Several iterations of the complete training
set to get a consistent set of weights that
works for all the training data


Computer Science Department FMIPA IPB 2003

Testing


Test the network after training


Examine network performance: measure the
network’s classification ability


Black box testing


Do the inputs produce the appropriate outputs?


Not necessarily 100% accurate


But may be better than human decision makers


Test plan should include


Routine cases


Potentially problematic situations


May have to retrain

Computer Science Department FMIPA IPB 2003

ANN Application Development
Process

1. Collect Data

2. Separate into Training and Test Sets

3. Define a Network Structure

4. Select a Learning Algorithm

5. Set Parameters, Values, Initialize Weights

6. Transform Data to Network Inputs

7. Start Training, and Determine and Revise
Weights

8. Stop and Test

9. Implementation: Use the Network with New Cases


Computer Science Department FMIPA IPB 2003

Data Collection and Preparation



Collect data and separate into a
training
set

and a
test set



Use
training cases

to adjust the weights



Use
test cases

for network validation


Computer Science Department FMIPA IPB 2003

Single Layer Perceptron

Computer Science Department FMIPA IPB 2003


Each pass through all of the training
input and target vector is called an
epoch
.


Computer Science Department FMIPA IPB 2003

Example :

Computer Science Department FMIPA IPB 2003

Computer Science Department FMIPA IPB 2003

Computer Science Department FMIPA IPB 2003

Disadvantage Perceptron


Perceptron networks can
only solve linearly
separable problems




see:Marvin Minsky and Seymour
Papert’s book
Perceptron [10]
.


[10] M.L. Minsky, S.A. Papert,
Perceptrons: An Introduction To Computational Geometry
,
MIT Press, 1969.

See XOR problem

Computer Science Department FMIPA IPB 2003

Multilayer Perceptrons (MLP)

Computer Science Department FMIPA IPB 2003

MLP


MLP has ability to learn complex
decision boundaries


MLPs are used in many practical
computer vision applications involving
classification (or supervised
segmentation).


Computer Science Department FMIPA IPB 2003

Backpropagation

Computer Science Department FMIPA IPB 2003

Computer Science Department FMIPA IPB 2003

X =
-
1 : 0.1 : 1;


Y = [
-
0.960
-
0.577
-
0.073 0.377 0.641 0.660 0.461...

0.134
-
0.201
-
0.434
-
0.500
-
0.393
-
0.165 0.099...

0.307 0.396 0.345 0.182
-
0.031
-
0.219
-
0.320];


Normalisasi :

pr = [
-
1 1]; m1 = 5; m2 = 1;


net_ff = newff (pr, [m1 m2], {'logsig' 'purelin'});


net_ff = init (net_ff); %Default Nguyen
-
Widrow initialization


%Training:

net_ff.trainParam.goal = 0.02;

net_ff.trainParam.epochs = 350;


net_ff = train (net_ff, X, Y);


%Simulation:

X_sim =
-
1 : 0.01 : 1;

Y_nn = sim (net_ff, X_sim);

Computer Science Department FMIPA IPB 2003

Backpropagation


Backpropagation (back
-
error propagation)


Most widely used learning


Relatively easy to implement


Requires
training data
for conditioning the
network before using it for processing other
data


Network
includes one or more hidden layers


Network is considered
a
feedforward

approach

Computer Science Department FMIPA IPB 2003


Externally provided correct patterns
are compared with the neural network
output during training (supervised
training)


Feedback adjusts the weights until all
training patterns are correctly
categorized

Computer Science Department FMIPA IPB 2003


Error is backpropogated through
network layers


Some error is attributed to each layer


Weights are adjusted


A large network can take a very long
time to train


May not converge

Computer Science Department FMIPA IPB 2003

Next Time …..



ANFIS Neural Network

By Ir. Agus Buono, M.Si, M.Komp