Robust Pareto Design of GMDH-type Neural Networks for Systems with Probabilistic Uncertainties

grandgoatAI and Robotics

Oct 23, 2013 (4 years and 17 days ago)

190 views

Robust Pareto Design of GMDH
-
type Neural
Networks for Systems with Probabilistic Uncertainties


N. Nariman
-
zadeh, F. Kalantary, A. Jamali, F. Ebrahimi

Faculty of Engineering,

The University of Guilan



System

identification

techniques

are

applied

in

many

fields

in

order

to

model

and

predict

the

behaviors

of

unknown

and/or

very

complex

systems

based

on

given

input
-
output

data


GMDH

is

a

self
-
organizing

approach

by

which

gradually

complicated

models

are

generated

based

on

the

evaluation

of

their

performances

on

a

set

of

multi
-
input
-
single
-
output

data


In

order

to

obtain

more

robust

models,

it

is

required

to

consider

all

the

conflicting

objectives,

namely,

training

error

(
TE
),

prediction

error

(
PE
)

in

the

sense

of

multi
-
objective

Pareto

optimization

process


For

multi
-
objective

optimization

problems,

there

is

a

set

of

optimal

solutions,

known

as

Pareto

optimal

solutions

or

Pareto

front


Introduction


System Identification Techniques Are Applied in Many Fields in
order to Model and Predict the Behaviors of Unknown and/or Very
Complex Systems Based on Given Input
-
Output Data.


Group Method of Data Handling (GMDH) Algorithm is Self
-
Organizing Approach by which Gradually Complicated Models are
Generated Based on the Evaluation of their Performances on a set
of Multi
-
Input
-
Single
-
Output Data Pairs (
i
=1, 2, …, M)

X
1

X
2

X
n

Y
1

.

.

Y
m

Modelling

Using GMDH
-
type Networks

The classical GMDH algorithm can be represented as set of neurons
in which different pairs of them in each layer are connected through
a quadratic polynomial and thus produce new neurons in the next
layer.

G
1

G
2

G
4

G
6

X
1

X
2

X
3

X
4


A Feedforward GMDH
-
Type Network


G
3

G
5

Input Layer

Output Layer

Hidden Layer(s)

Modelling

Using GMDH
-
type Networks


A Generalized GMDH Network Structure of a Chromosome

a

c

b

d

ad

bc

adbc

a d b c b c b c

Application of Genetic Algorithm in the Topology Design of GMDH
-
type NNs

a

c

b

d

ad

bc

adbc

a d b c d d d d

Application of Genetic Algorithm in the Topology Design of GMDH
-
type NNs


Crossover operation for two individuals in GS
-
GMDH networks

Application of Singular Value Decomposition

to the Design of GMDH
-
type Networks

SVD is the method for solving most linear least squares problems that
some singularities may exist in the normal equations

The

SVD

of

a

matrix,

,

is

a

factorization

of

the

matrix

into

the

product

of

three

matrices,

matrix

,

diagonal

matrix



with

non
-
negative

elements

(Singular

Values),

and

orthogonal

matrix

such

that

:

Genetic Algorithms and Multi
-
objective Pareto Optimization

Genetic algorithms are iterative
and stochastic optimization
techniques.

In the optimization of complex real
-
world
problems, there are several objective functions
to be optimized simultaneously.

There is
no single optimal solution

as the best
because objectives conflict each other.

There is a set of optimal solutions, well known
as
Pareto optimal solutions
or

Pareto front.

Modelling error

Prediction error

Multi
-
objective optimization

Modelling error

Prediction error

Multi
-
objective optimization

Difference between robust optimization and traditional optimization

Design Variable

Objective

Function

Feasible

Infeasible

Optimal solution

Robust optimal solution

Random variable

0.25

0.50

0.75

1.00

PDF

CDF

For the discrete sampling:

Stochastic Robust Analysis



Modelling

and

prediction

of

soil

shear

strength,

Su

,

based

on

5

input

parameters,

namely,

SPT

number

(Standard

Penetration

Test)

N

,

effective

overburden

stress

s
/
0
,

moisture

content

percent

W

,

LL

liquid

limit,

and

PL

plastic

limit

of

fine
-
graded

clay

soil



The

data

used

in

this

study

were

gathered

from

the

National

Iranian

Geotechnical

Database,

which

has

been

set

up

in

the

Building

and

Housing

Research

Centre

(BHRC)



The

database

has

been

established

under

a

mandate

from

the

Management

and

Planning

Organization

(MPORG),

which

supervises

the

professional

activities

of

all

of

the

consultancy

firms

in

Iran


Comparison of actual values with the evolved GMDH model corresponding to
optimum point C (nominal table)

Training set

Prediction set

Point

Network’s structure

TE

PE

Mean of TE

Mean of PE

Variance of TE

Variance of PE

A

bbaebcacbcaeacee

133.12

48.49

323.76

161.49

174862.64

42019.59

B

bcaebacdbcbbadde

79.20

260.15

73785.2

17844.7

3.8e11

3.3e9

C

bcaebccdbdbcaccd

89.79

75.30

28366.5

709.8

3.7e10

2.6e6

Objective functions and structure of networks of different optimum design points

Point

Network’s structure

TE

PE

Mean of
TE

Mean of PE

Variance of TE

Variance of PE

A

bbaebcacbcaeacee

133.12

48.49

323.76

161.49

174862.64

42019.59

B

bcaebacdbcbbadde

79.20

260.15

73785.2

17844.7

3.8e11

3.3e9

C

bcaebccdbdbcaccd

89.79

75.30

28366.5

709.8

3.7e10

2.6e6

D

abeecddd

132.79

237.59

234 .61

248.03

178.77

1174.283

Objective functions and structure of networks of different optimum design points

Y
1

Y
4

Y
3

Y
5

Y
2

Point C

Point D

The structure of network corresponding to point C and D

Y
1
=
-
5.94
+
0.65
N


+
0.76
σ
0


-
0.0083
N

2

-

0.0019
σ
0

2

+
0.0013
N


σ
0



Y
2
=
25.42
-

2.76
w +
1.86
LL
-

0.019
w
2

-

0.045
LL
2

+
0.11
w(LL)


Y
3
=

16.99
+
0.82
Y
2
-

1.27
LL
-

0.0015
Y
2
2
+
0.016
(LL)
2

+
0.015
(Y
2
)(LL)


Y
4
=

10.16
+
0.74
Y
1
-

0.22
PL
-

0.019
Y
1
2

-

0.034
PL
2

+
0.056
(Y
1
)(PL)


Y
5
=

16.12
+
0.83
Y
4
-

0.64
Y
3
-

0.0004
Y
4
2

+
0.0060
Y
3
2
+
0.0036
(Y
4
)(Y
3
)


Conclusion


A multi
-
objective genetic algorithm was used to optimally
design GMDH
-
type neural networks from a robustness point
of view in a probabilistic approach.


Multi
-
objective optimization of robust GMDH models led to
the discovering some important trade
-
off among those
objective functions.


The framework of this work is very promising and can be
generally used in the optimum design of GMDH models in
real
-
world complex systems with probabilistic uncertainties.

Thanks for your attention…