Robust Pareto Design of GMDH

type Neural
Networks for Systems with Probabilistic Uncertainties
N. Nariman

zadeh, F. Kalantary, A. Jamali, F. Ebrahimi
Faculty of Engineering,
The University of Guilan
•
System
identification
techniques
are
applied
in
many
fields
in
order
to
model
and
predict
the
behaviors
of
unknown
and/or
very
complex
systems
based
on
given
input

output
data
•
GMDH
is
a
self

organizing
approach
by
which
gradually
complicated
models
are
generated
based
on
the
evaluation
of
their
performances
on
a
set
of
multi

input

single

output
data
•
In
order
to
obtain
more
robust
models,
it
is
required
to
consider
all
the
conflicting
objectives,
namely,
training
error
(
TE
),
prediction
error
(
PE
)
in
the
sense
of
multi

objective
Pareto
optimization
process
•
For
multi

objective
optimization
problems,
there
is
a
set
of
optimal
solutions,
known
as
Pareto
optimal
solutions
or
Pareto
front
Introduction
•
System Identification Techniques Are Applied in Many Fields in
order to Model and Predict the Behaviors of Unknown and/or Very
Complex Systems Based on Given Input

Output Data.
•
Group Method of Data Handling (GMDH) Algorithm is Self

Organizing Approach by which Gradually Complicated Models are
Generated Based on the Evaluation of their Performances on a set
of Multi

Input

Single

Output Data Pairs (
i
=1, 2, …, M)
X
1
X
2
X
n
Y
1
.
.
Y
m
Modelling
Using GMDH

type Networks
The classical GMDH algorithm can be represented as set of neurons
in which different pairs of them in each layer are connected through
a quadratic polynomial and thus produce new neurons in the next
layer.
G
1
G
2
G
4
G
6
X
1
X
2
X
3
X
4
A Feedforward GMDH

Type Network
G
3
G
5
Input Layer
Output Layer
Hidden Layer(s)
Modelling
Using GMDH

type Networks
A Generalized GMDH Network Structure of a Chromosome
a
c
b
d
ad
bc
adbc
a d b c b c b c
Application of Genetic Algorithm in the Topology Design of GMDH

type NNs
a
c
b
d
ad
bc
adbc
a d b c d d d d
Application of Genetic Algorithm in the Topology Design of GMDH

type NNs
Crossover operation for two individuals in GS

GMDH networks
Application of Singular Value Decomposition
to the Design of GMDH

type Networks
SVD is the method for solving most linear least squares problems that
some singularities may exist in the normal equations
The
SVD
of
a
matrix,
,
is
a
factorization
of
the
matrix
into
the
product
of
three
matrices,
matrix
,
diagonal
matrix
with
non

negative
elements
(Singular
Values),
and
orthogonal
matrix
such
that
:
Genetic Algorithms and Multi

objective Pareto Optimization
Genetic algorithms are iterative
and stochastic optimization
techniques.
In the optimization of complex real

world
problems, there are several objective functions
to be optimized simultaneously.
There is
no single optimal solution
as the best
because objectives conflict each other.
There is a set of optimal solutions, well known
as
Pareto optimal solutions
or
Pareto front.
Modelling error
Prediction error
Multi

objective optimization
Modelling error
Prediction error
Multi

objective optimization
Difference between robust optimization and traditional optimization
Design Variable
Objective
Function
Feasible
Infeasible
Optimal solution
Robust optimal solution
Random variable
0.25
0.50
0.75
1.00
PDF
CDF
For the discrete sampling:
Stochastic Robust Analysis
•
Modelling
and
prediction
of
soil
shear
strength,
Su
,
based
on
5
input
parameters,
namely,
SPT
number
(Standard
Penetration
Test)
N
′
,
effective
overburden
stress
s
/
0
,
moisture
content
percent
W
,
LL
liquid
limit,
and
PL
plastic
limit
of
fine

graded
clay
soil
•
The
data
used
in
this
study
were
gathered
from
the
National
Iranian
Geotechnical
Database,
which
has
been
set
up
in
the
Building
and
Housing
Research
Centre
(BHRC)
•
The
database
has
been
established
under
a
mandate
from
the
Management
and
Planning
Organization
(MPORG),
which
supervises
the
professional
activities
of
all
of
the
consultancy
firms
in
Iran
Comparison of actual values with the evolved GMDH model corresponding to
optimum point C (nominal table)
Training set
Prediction set
Point
Network’s structure
TE
PE
Mean of TE
Mean of PE
Variance of TE
Variance of PE
A
bbaebcacbcaeacee
133.12
48.49
323.76
161.49
174862.64
42019.59
B
bcaebacdbcbbadde
79.20
260.15
73785.2
17844.7
3.8e11
3.3e9
C
bcaebccdbdbcaccd
89.79
75.30
28366.5
709.8
3.7e10
2.6e6
Objective functions and structure of networks of different optimum design points
Point
Network’s structure
TE
PE
Mean of
TE
Mean of PE
Variance of TE
Variance of PE
A
bbaebcacbcaeacee
133.12
48.49
323.76
161.49
174862.64
42019.59
B
bcaebacdbcbbadde
79.20
260.15
73785.2
17844.7
3.8e11
3.3e9
C
bcaebccdbdbcaccd
89.79
75.30
28366.5
709.8
3.7e10
2.6e6
D
abeecddd
132.79
237.59
234 .61
248.03
178.77
1174.283
Objective functions and structure of networks of different optimum design points
Y
1
Y
4
Y
3
Y
5
Y
2
Point C
Point D
The structure of network corresponding to point C and D
Y
1
=

5.94
+
0.65
N
’
+
0.76
σ
0
’

0.0083
N
’
2

0.0019
σ
0
’
2
+
0.0013
N
’
σ
0
’
Y
2
=
25.42

2.76
w +
1.86
LL

0.019
w
2

0.045
LL
2
+
0.11
w(LL)
Y
3
=
16.99
+
0.82
Y
2

1.27
LL

0.0015
Y
2
2
+
0.016
(LL)
2
+
0.015
(Y
2
)(LL)
Y
4
=
10.16
+
0.74
Y
1

0.22
PL

0.019
Y
1
2

0.034
PL
2
+
0.056
(Y
1
)(PL)
Y
5
=
16.12
+
0.83
Y
4

0.64
Y
3

0.0004
Y
4
2
+
0.0060
Y
3
2
+
0.0036
(Y
4
)(Y
3
)
Conclusion
•
A multi

objective genetic algorithm was used to optimally
design GMDH

type neural networks from a robustness point
of view in a probabilistic approach.
•
Multi

objective optimization of robust GMDH models led to
the discovering some important trade

off among those
objective functions.
•
The framework of this work is very promising and can be
generally used in the optimum design of GMDH models in
real

world complex systems with probabilistic uncertainties.
Thanks for your attention…
Comments 0
Log in to post a comment