(Back-Propagation Neural Network)

trainerhungarianΤεχνίτη Νοημοσύνη και Ρομποτική

20 Οκτ 2013 (πριν από 3 χρόνια και 5 μήνες)

64 εμφανίσεις

Artificial Neural Network

(Back
-
Propagation Neural Network)

Yusuf
Hendrawan
, STP.,
M.App.Life

Sc.,
Ph.D

Neurons

http://faculty.washington.edu/chudler/color/pic1an.gif

http://research.yale.edu/ysm/images/78.2/articles
-
neural
-
neuron.jpg

Biologica
l

Artificial

A typical AI agent

Neural Network Layers


Each layer
receives its
inputs from the
previous layer
and forwards
its outputs to
the next layer

http://smig.usgs.gov/SMIG/features_0902/tualatin_ann.fig3.gif

Multilayer feed
forward
network

It contains one or more hidden layers
(hidden neurons).

“Hidden” refers to the part of the
neural network is not seen directly
from either input or output of the
network .

The function of hidden neuron is to
intervene between input and output.

By adding one or more hidden layers,
the network is able to extract higher
-
order statistics from input

Neural Network Learning

Back
-
Propagation Algorithm:

function

BACK
-
PROP
-
LEARNING(
examples, network
)
returns

a neural network


inputs:

examples
, a set of examples, each with input vector
x

and output vector
y


network
, a multilayer network with
L

layers, weights
W
j,i

,
activation
function

g



repeat


for each
e

in

examples

do


for each
node
j

in the input layer
do

a
j



x
j
[
e
]


for
l

= 2
to

M

do


in
i



j

W
j,i

a
j


a
i



g
(
in
i
)


for each
node
i

in the output layer
do


D
j



g

(
in
j
)


i

W
ji

D
i



for
l

=
M



1
to

1
do


for each
node
j

in layer
l
do


D
j



g’
(
in
j
)


i

W
j,i

D
i


for each
node
i

in layer
l

+ 1
do


W
j,i



W
j,i

+
a
x

a
j

x

D
i


until
some stopping criterion is satisfied


return
NEURAL
-
NET
-
HYPOTHESIS(
network
)


[Russell, Norvig] Fig. 20.25 Pg. 746

Back
-
Propagation Illustration

ARTIFICIAL NEURAL NETWORKS Colin Fahey's Guide (Book CD)

X
1

X2

Y

Vo

Wo

Z1

Z2

Z
3

Z
4

Input (X)

Hidden

Output (Y)

X1

X2

0.3

0.4

0.5

0.6

0.2

0.3

0.4

0.7

Input (X)

Output / Target (T)

T

0.1

0.8

0.4

0.5

Jumlah

Neuron
pada

Input Layer

2

Jumlah

Neuron

pada

Hidden Layer

4

Jumlah

Neuron

pada

Output Layer

1

Learning rate (
α
)

0.1

Momentum (m)

0.9

Target

Error

0.01

Maximum Iteration

1000

Bobot

Awal

Input
ke

Hidden

V11 = 0.75

V21 = 0.35

V12 = 0.54

V22 = 0.64

V13 = 0.44

V23 = 0.05

V14 = 0.32

V24 = 0.81

Bias
ke

Hidden

Vo
11

= 0.07

Vo
21

= 0.12

Vo
12

= 0.91

Vo
22

= 0.23

Vo
13

= 0.45

Vo
23

= 0.85

Vo
14

= 0.25

Vo
24
= 0.09

Bobot

Awal

Hidden
ke

Output

W1 = 0.04

W2 = 0.95

W3 = 0.33

W4 = 0.17

Bias
ke

Output

Wo
1

= 0.66

Wo
2

= 0.56

Wo
3

= 0.73

Wo
4

= 0.01

Menghitung

Zin

&
Z

dari

input
ke

hidde
n

Z
in
(1)

= (X1 * V11) + (X2 * V21) = (0.3 * 0.75) + (0.4 * 0.35) = 0.302

Z
in
(2)

= (X1 * V12) + (X2 * V22) = (0.3 * 0.54) + (0.4 * 0.64) = 0.418

Z
in
(3)

= (X1 * V13) + (X2 * V23) = (0.3 * 0.44) + (0.4 * 0.05) = 0.152

Z
in
(4)

= (X1 * V14) + (X2 * V24) = (0.3 * 0.32) + (0.4 * 0.81) = 0.42

Menghitung

Yin

&
Y

dari

hidden
ke

output

Y
in


= (Z
(
1
)

* W
1
) +
(
Z
(
2
)

*
W
2
) +

(
Z
(
3
)

*
W
3
) +

(
Z
(
4
)

*
W
4
)


= (
0.57
*
0.04
) + (
0.603
*
0.95
) + (
0.538
*
0.33
) + (
0.603
*
0.17
)


=
0.876

Menghitung

dev
antara


Y

dengan

output
nyata

dev = (T
-

Y) * Y * (1
-

Y) = (0.1


0.706) * 0.706 * (1


0.706) =
-
0.126

Menghitung

selisih

selisih

= T
-

Y=
-
0.606

Back
-
Propagation


Menghitung

d
in

dari

output
ke

hidden

d
in(1)

= (dev * W1) = (
-
0.126 * 0.04) =
-
0.00504

d
in(
2
)

= (dev * W
2
) = (
-
0.126
*
0.95
) =
-
0.1197

d
in(
3
)

= (dev * W
3
) = (
-
0.126
*
0.33
) =
-
0.04158

d
in(4)

= (dev * W4) = (
-
0.126 * 0.17) =
-
0.02142

Menghitung

d

d

(1)

= (
d
in(1
)

*

Z
(1)

* (1
-

Z
(1)

) = (
-
0.00504 * 0.575 * (1


0.575) =
-
0.00123

d

(2)

= (d
in(2)

*

Z
(2)

* (1
-

Z
(2)

) = (
-
0.1197 * 0.603 * (1


0.603) =
-
0.02865

d

(3)

= (d
in(3)

*

Z
(3)

* (1
-

Z
(3)

) = (
-
0.04158 * 0.538 * (1


0.538) =
-
0.01033

d

(4)

= (d
in(4)

*

Z
(4)

* (1
-

Z
(4)

) = (
-
0.02142 * 0.603 * (1


0.603) =
-
0.00512

Mengkoreksi

bobot

(W)
dan

bias (
Wo
)

W
1
= W
1
+ (
α

* dev *
Z
(
1
)

) + (m *
Wo
(
1
)
) =
0.04
+ (
0.1
*
-
0.126
*
0.575
) + (
0.9
*
0.66
)



=
0.627

W2 = W2 + (
α

* dev * Z
(2)

) + (m *
Wo
(2)
) = 0.95 + (0.1 *
-
0.126 * 0.603) + (0.9 * 0.56)



= 1.45

W3 = W3 + (
α

* dev * Z
(3)

) + (m *
Wo
(3)
) = 0.33 + (0.1 *
-
0.126 * 0.538) + (0.9 * 0.73)



= 0.98

W4 = W4 + (
α

* dev * Z
(4)

) + (m *
Wo
(4)
) = 0.17 + (0.1 *
-
0.126 * 0.603) + (0.9 * 0.01)



= 0.171

Wo
1

= (
α

* Z
(1
)

) + (m *
Wo
(1)
) = (0.1 * 0.575) + (0.9 * 0.66) = 0.65

Wo
2
= (
α

* Z
(2)

) + (m *
Wo
(2)
) = (0.1 * 0.603) + (0.9 * 0.56) = 0.564

Wo
3
= (
α

* Z
(3)

) + (m *
Wo
(3)
) = (0.1 * 0.538) + (0.9 * 0.73) = 0.71

Wo
4
= (
α

* Z
(
4
)

) + (m *
Wo
(
4
)
) = (
0.1
*
0.603
) + (
0.9
*
0.01
) =
0.0693

Mengkoreksi

bobot

(V)
dan

bias (Vo)

V11 = V11 + (
α

*
d

(1)

* X1 ) + (m * Vo
(11)
) = 0.75 + (0.1 *
-
0.00123 * 0.3) + (0.9 * 0.07)



= 0.8129

V12 = V12 + (
α

*
d

(2)

* X1 ) + (m * Vo
(12)
) = 0.54 + (0.1 *
-
0.02865 * 0.3) + (0.9 * 0.91)



= 1.3581

V13 = V13 + (
α

*
d

(3)

* X1 ) + (m * Vo
(13)
) = 0.44 + (0.1 *
-
0.01033 * 0.3) + (0.9 * 0.45)



= 0.8446

V14 = V14 + (
α

*
d

(4)

* X1 ) + (m * Vo
(14)
) = 0.32 + (0.1 *
-
0.00512 * 0.3) + (0.9 * 0.25)



= 0.5448

V21 = V21 + (
α

*
d

(1)

* X2 ) + (m * Vo
(21)
) = 0.35 + (0.1 *
-
0.00123 * 0.4) + (0.9 * 0.12)



= 0.4579

V22 = V22 + (
α

*
d

(2)

* X2 ) + (m * Vo
(22)
) = 0.64 + (0.1 *
-
0.02865 * 0.4) + (0.9 * 0.23)



= 0.8458

V
23
= V
23
+ (
α

*
d

(
3
)

* X
2
) + (m * Vo
(
23
)
) =
0.05
+ (
0.1
*
-
0.01033
*
0.4
) + (
0.9
*
0.85
)



=
0.8145

V
24
= V
24
+ (
α

*
d

(
4
)

* X
2
) + (m * Vo
(
24
)
) =
0.81
+ (
0.1
*
-
0.00512
*
0.4
) + (
0.9
*
0.09
)



=
0.8907

Mengkoreksi

bobot

(V)
dan

bias (Vo)

Vo
11

= (
α

* d

(1)

* X1 ) + (m * Vo
11
) = (0.1 *
-
0.00123*0.3)+(0.9*0.07) = 0.0629

Vo
12

= (
α

* d

(2)

* X1 ) + (m * Vo
12
) = (0.1 *
-
0.02865*0.3)+(0.9*0.91) = 0.8181

Vo
13

= (
α

* d

(3)

* X1 ) + (m * Vo
13
) = (0.1 *
-
0.01033*0.3)+(0.9*0.45) = 0.4046

Vo
14

= (
α

* d

(4)

* X1 ) + (m * Vo
14
) = (0.1 *
-
0.00512*0.3)+(0.9*0.25) = 0.2248

Vo
21

= (
α

* d

(1)

* X2 ) + (m * Vo
21
) = (0.1 *
-
0.00123*0.4)+(0.9*0.12) = 0.1079

Vo
22

= (
α

* d

(
2
)

* X
2
) + (m * Vo
22
) = (
0.1
*
-
0.02865
*
0.4
)+(
0.9
*
0.23
) =
0.2058

Vo
23

= (
α

* d

(
3
)

* X
2
) + (m * Vo
23
) = (
0.1
*
-
0.01033
*
0.4
)+(
0.9
*
0.85
) =
0.7645

Vo
24

= (
α

* d

(4)

* X2 ) + (m * Vo
24
) = (0.1 *
-
0.00512*0.4)+(0.9*0.09) = 0.0807