GMDH-type neural network

runmidgeΤεχνίτη Νοημοσύνη και Ρομποτική

20 Οκτ 2013 (πριν από 3 χρόνια και 8 μήνες)

58 εμφανίσεις

GMDH
-
type neural
networks

1

GMDH


Inductive Learning Algorithms for Complex Systems Modeling


Alexey
G.
Ivakhnenko

(
1994
)



Group Method of Data Handling


Inductive learning


“Survival of the fittest”


Fully automatic structural


“Self
-
organization”


“external criterion”


Parametric optimization

2

Unit neuron


Polynomial neural networks


Solving the nonlinear problem using polynomial instead of linear
combination.


Kolmogorov
-
Gabor polynomial

y
=
𝑎
0
+

𝑎



𝑛

=
1
+


𝑎





+



𝑎







𝑛

=
1
𝑛

=
1
𝑛

=
1
𝑛

=
1
𝑛

=
1
+

,




=
𝑓


,


=

0
+

1


+

2


+

3




+

4


2
+

5


2
,

3










𝑓


,




Weight update:


XW
=
y



𝑇

=

𝑇




=
(

𝑇

)

1

𝑇




=
1
,


,

,


2
,

=
(

0
,

1
,

,

5
)
𝑇



W
=
α
y

y


𝑇


Back propagation:

1
-
hidden
-
layer:f(XW)

2
-
hidden
-
layer:f(f(XU)W)

4

U

W

Error
ε


u
=

α
𝜕
ε
𝜕
u
=

𝛼

𝜕


𝑓
𝑓
𝑈

𝜕𝑓
𝑓
𝑈

𝜕𝑓
𝑓
𝑈

𝜕𝑓
𝑈
𝜕𝑓
𝑈
𝜕𝑢
=
𝛼



𝑓
(
𝑓
(
𝑈
)

)
𝜕𝑓
(
𝑓
(
𝑈
)

)
𝜕𝑓
(
𝑈
)


External Criterion


Principle of self
-
organization








Regularity criterion


It consists of a squared error from testing set B


2

=

2

/

=





(

)

2
=








2


Symmetric Regularity criterion


2

=

2

/

+

2

/

=








2
+








2


5

complexity

delta

Δ
(B)

ε
(A+B)

A: training set

B: testing set

Δ(): external criterion

Global minimum

Initial model selection

6

External criterion

Optimal estimation