20

th

European Symposium on Computer Aided Process Engineering – ESCAPE20

S. Pierucci and G. Buzzi Ferraris (Editors)

© 2010 Elsevier B.V. All rights reserved.

A Combination of Parallel Computing and Object-

Oriented Programming to Improve Optimizer

Robustness and Efficiency

Guido Buzzi-Ferraris, Flavio Manenti

Dipartimento di Chimica, Materiali e Ingegneria Chimica “Giulio Natta”

Politecnico di Milano, Piazza Leonardo da Vinci 32, 20133 Milano, ITALY

E-mail: guido.buzziferraris@polimi.it

Abstract

This research activity is mainly aimed at showing potentialities in coupling object-

oriented programming with parallel computing. Wide margins of benefits could be

obtained in algorithm efficiency and robustness with a relative small programming

effort. The case of unconstrained multi-dimensional optimization is proposed as

quantitative example.

Keywords: Robust optimization, Multimodality, Narrow valleys, Discontinuous

functions, Parallel Computing.

1. Introduction

We are undergoing two silent revolutions that directly involve Process Systems

Engineering (PSE) and Computer-Aided Process Engineering (CAPE) communities,

besides many other scientific and industrial areas: the object-oriented programming and

the parallel computing on personal computer. Both these transformations have been

widely discussed in the literature as they significantly modify the numerical analysis as

it was conceived since the second part of the previous century and the way to apply

numerical methods and algorithms for solving more and more complex problems and

multifaceted issues.

Nevertheless, since it is not clearly stated in the current literature, it is worth remarking

that the parallel computing is easy to integrate in object-oriented programming and their

combination seems particularly appealing as many objects generated by the same class

might run simultaneously on different processors or cluster nodes. By thinking parallel

and object-oriented both together, it is possible to write by new many algorithms which

were not considered for solving numerical problems because of their reduced

performances in the procedural programming and sequential computing.

Thus, this research activity specifically deals with the development of very robust

optimizers that exploit:

• All features of object-oriented programming (Buzzi-Ferraris, 1994), which allow

going beyond the procedural programming and its limitations.

• The shared memory nowadays commonly available on multi-processor machines

(distributed memory machines are not considered for the time being, even though

the same reasoning here described can be extended to this branch of parallel

computing).

Motivation and practical interests in some scientific and industrial areas are briefly

reported in Paragraph 2. Basic concepts of coupling parallel computing with object-

G. Buzzi-Ferraris and F. Manenti

oriented programming for improving the optimizer robustness and efficiency are stated

in Paragraph 3. Some literature tests involving multidimensionality, strong and weak

multimodality, very narrow valleys, and functions that are undefined in some regions

are proposed in Paragraph 4.

2. Motivation and Practical Interests

Looking at the increasing spread of multi-processor machines as well as the larger and

larger amount of processors available on the common PCs, it is easy to see how the

period we are living is very similar to the one of 1970s when the most powerful (and

very large-size) machines were gradually replaced by smallest personal computers with

reduced computational power, but with a large impact on research activities for their

faster spread, reduced costs, reasonably good performances, and especially for their

higher slope in innovation. In our opinion, but it is easy to find out some confirmations

yet, shared memory machines will have a faster evolution than distributed memory

architectures and looking forward this, we preferred to use openMP directives rather

than MPI ones in starting exploiting parallel computing, at least for this preliminary

research activity.

In any case, the result is practically the same by using one or the other set of directives

as efficiency and robustness of many algorithms can be significantly improved so to

increase performance solution of a series of industrial issues and to allow moving from

reacting to predicting technologies applied to industrial plants (Manenti, 2009) and from

those solutions still performed off-line to their on-line application (White, 2001) by

preserving the robustness of the selected methods.

To quote some examples typical of process industry and PSE/CAPE communities,

improvements in optimizer efficiency and robustness can provide practical benefits in

data reconciliation (Arora and Biegler, 2001; Bagajewicz, 2003), in data regression

(Buzzi-Ferraris and Manenti, 2009a, 2009b, 2010b; Manenti and Buzzi-Ferraris, 2009),

in solving nonlinear algebraic or differential systems (Cuoci et al., 2007; Manenti et al.,

2009), or in the supply chain management optimization levels (Dones et al., 2009; Lima

et al., 2009; Manenti and Rovaglio, 2008).

3. Exploiting Shared Memory to Improve Efficiency and Robustness

Conventional programs easily fail when some specific families of optimization

problems have to be solved. Very robust optimizers are required in these cases:

• When the function is multimodal and the global optimum is required

• The function and/or its derivatives are discontinuous

• The function cannot be approximated by a quadric in correspondence with the

optimum

• Very narrow valleys (or steep walls) are present

• The function is undefined in some regions and the domain cannot be analytically

described

Although no one can ensure the global optimum is found, a robust algorithm should be

effective in tackling all previous situations.

Let us start analyzing the problem of very narrow valleys. From this perspective, the

OPTNOV’s method (Buzzi-Ferraris, 1967) seems one of the most appealing approach.

It is important to realize the reason that makes traditional methods such as Simplex

(Nelder and Mead, 1965), Hooke-Jeeves (Hooke and Jeeves, 1961), Rosembrock

(Rosenbrock, 1960), Quasi-Newton algorithms and so on ineffective when the function

valleys are particularly narrow. For example, Rosembrock’s method is based on the

A Combination of Parallel Computing and Object-Oriented Programming to Improve

Optimizer Robustness and Efficiency

rotation of axes of search so to follow the bottom of the valley. Since one of rotated

axes is adopted as search direction, it may occur that moving along it does not bring to

any function improvement when the valley is very narrow as shown in Figure 1.

x

1

x

2

Figure 1. Rosembrock’s method fails with very narrow valleys.

To exploit the search direction that inaccurately detects the bottom of the valley, it is

necessary to change the point of view. OPTNOV’s method is based on some simple

ideas that make it particularly robust and efficient in the case of very narrow valleys:

• Whatever optimization algorithm is able to find the bottom of the valley by

starting from a point outside the same valley

• The line joining two points on the bottom of the valley is a reasonable valley

direction; therefore a point projected along such a direction has good probabilities

to be close to the valley

• Nevertheless, this valley direction must not be used as direction of one-

dimensional search, rather as a direction which a new point projection must be

carried out along

• This new point should not be discarded even though it is worse than the previous

one, rather it is the new starting point for the search.

• This search must be performed in the sub-space orthogonal to the valley direction

to prevent the problem of having small steps

This philosophy is particularly effective in an object-oriented programming coupled

with parallel computing as many reduced optimizations must be carried out starting

from distinct points and they can be independently solved each other. Consequently,

this philosophy of simultaneously solving different optimization problems by starting

from distinct guesses allows rationally facing even the global minimum paradigm.

The concept to build up a program for effectively tackling all aforementioned issues is

rather trivial as it is possible to develop an optimizer consisting of

N

objects, where

N

is the number of available processors and each of them uses in turn an optimizer

reasonably robust.

Hence, two distinct problems must be solved: the first is the selection of points used in

the

N

objects as initial guess and the second is which optimizer to use within each of

these

N

objects; it is worth remarking that even this optimizer must be opportunely

robust: to manage possible first- and second-order discontinuities of the function; to

overcome possible regions where the same function is undefined; to ensure the global

minimum in one-dimensional searches is found; and to efficiently tackle the problem of

slightly narrow valleys.

For the sake of clarity, let us call inner the optimizer used within each of the

N

objects

and outer the one managing the overall optimization problem. The outer optimizer only

is discussed in this paper.

G. Buzzi-Ferraris and F. Manenti

The problem of searching for the global optimum of the overall problem and to

overcome its possible narrow valleys are both tasks of the outer optimizer. The

following strategy is proposed to manage the

N

objects: three objects are required for

applying OPTNOV’s philosophy whereas the remaining

3N

−

objects are selected by

using the same techniques employed in optimal experimental design (Buzzi-Ferraris,

1999; Buzzi-Ferraris and Manenti, 2009a, 2010a, 2010b; Manenti and Buzzi-Ferraris,

2009). Therefore, there is the lower bound of using four processors (QUAD CORE

machines). The outer optimizer collects initial and arrival points of each inner object

and it selects the two points having the best performances among all those ones

collected. If these two points are significantly close, the best third, fourth… is selected

in spite of the second to avoid any ill-conditioning while detecting the valley direction.

B

I

A

II

III

δ

δ

ε

B

I

A

II

III

δ

δ

ε

Figure 2. Points A and B are on the bottom of the valley (A is the best one); points I, II, and III

are the possible point projections along the valley direction

Distances

δ

湤=

ε

=潮朠灯p湴猠捡渠扥n摵捥搠潲硰≤湤n搠慣捯牤楮朠瑯⁴桥敳畬瑳a=

景爠r•慭灬攬f⁴h攠灯p湴⁉IIri湧n⁴o整ter=inner optimum, distances are expanded.

Points from the fourth to the

N th

−

are selected so to have the farthest points against

all the collected ones. This selection is efficiently carried out by using those techniques

adopted and proven for the optimal design of experiments. The following procedure is

adopted as stop criterion. At each iteration, the number of points in the neighborhood of

the optimum (given a tolerance value) is checked. If such a number is reasonable

(according to an assigned value), a possible solution is reached. Theoretically, the

number of points should be in the order of magnitude of the optimization problem

dimensions, but it is preferable to use smaller numbers when the optimization size is

large.

4. Numerical Tests

Many numerical tests were carried out to check the algorithm robustness for problems

of different dimensions. Tests of Table 1 are well-known literature functions for:

•

“Strong” multimodality issues: all directions are directions of function increase in

correspondence with local minima. Both Rastrigin (1) and Haupt (2) functions

were adopted and reported in Figure 3.

•

“Weak” multimodality issues: the function is constant along at least one direction

in correspondence of some local minima. Michalewicz’s function (3) was adopted

and reported in Figure 3.

•

Discontinuities and regions where the function is not defined (4). Figure 4 shows

the function against the variable

1

x

for the optimal value of

2

x

.

•

Extremely narrow valleys: valleys consisting of steep walls make the search of

the minimum a problematic issue for many optimizers. Buzzi-Ferraris’s function

(5) is adopted and reported in Figure 4.

( )

( )

2

1

110 10cos 2

n

RASTRIGIN i i

i

F

n x xπ

=

= − ⋅ + −

∑

(1)

A Combination of Parallel Computing and Object-Oriented Programming to Improve

Optimizer Robustness and Efficiency

( )

( )

4

1

, 0

where: sin 2

0, 0

HAUPT

i i

i

HAUPT

F a a

a x x

F a

π

=

= − >

⎧

=

⎨

= ≤

⎩

∏

(2)

( )

2

2

1

sin sin

n

n

i

MICHALEWICZ i

i

i x

F x

π

⋅

=

⎛ ⎞

⎛ ⎞

⎛ ⎞

⋅

⎜ ⎟

= − ⋅

⎜ ⎟

⎜ ⎟

⎜ ⎟

⎜ ⎟

⎝ ⎠

⎝ ⎠

⎝ ⎠

∑

(3)

( )

( )

( )

1

,1 1 1 1 1

2 1 1

945 1689 950 230 25

10 10 10 6

x

BUZZI FERRARIS A

F x x x x x e

x x x

−

−

⎡ ⎤

= − + + − + + − + + +

⎣ ⎦

+ − + −

(4)

( )( )( )( )( ) ( )

2

2

,2 1 1 1 1 1 1

10000 1 3 5 7 9 8

BUZZI FERRARIS B

F x x x x x x x

−

= − − − − − − + −

⎡ ⎤

⎣ ⎦

(5)

HAHAUPT

Hau

p

t

-10

-5

0

5

10

-10

-5

0

5

10

0

50

100

150

200

250

Rastrigin

0

1

2

3

4

0

1

2

3

4

0

0.5

1

1.5

2

Michalewicz

Figure 3. Two-dimensional Haupt (left), Rastrigin (middle), and Michalewicz (right) functions

Solution

x1:_8.00000000000

x2:–1.050000e+006

_y:_1.092632e-023

Starting point

x1:

_

1.000000000

x2:

_

0.000000000

_

y:

_

49.00000000

10

0

-10

7

+10

7

Figure 4. Buzzi-Ferraris’s functions (A, left; B, right) to test optimizer robustness

Table 1. Optimization tests

Starting point Number of iterations Optimum value

Rastrigin

2n =

0

8.

=

x

2414 -240

Rastrigin

10n =

0

8.

=

x

8357 -1200

Haupt

0

0.

=

x

5559 -19.8630560097267

Michalewicz

2n =

0

3.

=

x

1683 -1.80130340983985

Michalewicz

10n =

0

3.

=

x

20144 -9.660152

Buzzi-Ferraris A

{

}

0

1.;1.=x

16714 6.708389

Buzzi-Ferraris B

{

}

0

1.;0.=x

1085 1.009e-018

G. Buzzi-Ferraris and F. Manenti

5. Conclusions and Future Developments

This preliminary research activity shows the way and reports some benefits coming

from the interaction of parallel computing and object-oriented programming.

Specifically, the example of C++ class for robust optimization that could generate a

series of objects so that each of them could run on a specific processor by increasing the

same optimizer robustness with a small programming effort is proposed.

References

Arora, N., L.T. Biegler. (2001). Redescending estimators for data reconciliation and parameter

estimation. Computers & Chemical Engineering, 25(11-12), 1585-1599.

Bagajewicz, M.J. (2003). Data Reconciliation and Instrumentation Upgrade. Overview and

Challenges. FOCAPO 2003. 4th International Conference of Computer-Aided Process

Operations, Coral Springs, Florida, 103-116.

Buzzi-Ferraris, G. (1967). Ottimizzazione di funzioni a più variabili. Nota I. Variabili non

vincolate. Ing. Chim. It., 3, 101.

Buzzi-Ferraris, G. (1994). Scientific C++. Building Numerical Libraries, the Object-Oriented

Way. 2nd Ed., 479pp, Addison-Wesley, Cambridge University Press, ISBN 0-201-63192-X.

Buzzi-Ferraris, G. (1999). Planning of experiments and kinetic analysis. Catalysis Today, 52,

125-132.

Buzzi-Ferraris, G., F. Manenti. (2009a). Kinetic models analysis. Chemical Engineering Science,

64(5), 1061-1074.

Buzzi-Ferraris, G., F. Manenti. (2009b). Outlier Detection. Computers & Chemical Engineering,

submitted.

Buzzi-Ferraris, G., F. Manenti. (2010a). Fundamentals and Linear Algebra for the Chemical

Engineer Solving Numerical Problems. ISBN: 978-3-527-32552-8, WILEY-VCH, Weinheim.

Buzzi-Ferraris, G., F. Manenti. (2010b). Interpolation and Regression Models for the Chemical

Engineer Solving Numerical Problems. ISBN: 978-3-527-32652-5, WILEY-VCH, Weinheim.

Cuoci, A., A. Frassoldati, G. Buzzi-Ferraris, T. Faravelli, E. Ranzi. (2007). The ignition,

combustion and flame structure of carbon monoxide/hydrogen mixtures. Note 2: Fluid

dynamics and kinetic aspects of syngas combustion. International Journal of Hydrogen

Energy, 32(15), 3486-3500.

Dones, I., F. Manenti, H.A. Preisig, G. Buzzi-Ferraris. (2009). Nonlinear Model Predictive

Control: a Self-Adaptive Approach. Industrial & Engineering Chemistry Research, to appear.

Hooke, R., T.A. Jeeves. (1961). "Direct Search" Solution of Numerical and Statistical Problems.

J. of the Assn. For Computing Machinery, 8, 212-229.

Lima, N.M.N., F. Manenti, R. Maciel Filho, M. Embiruçu, M.R. Wolf Maciel. (2009). Fuzzy

Model-Based Predictive Hybrid Control of Polymerization Processes. Industrial &

Engineering Chemistry Research, 48(18), 8542–8550.

Manenti, F. (2009). From Reacting to predicting technologies: A novel performance monitoring

technique based on detailed dynamic models. Chemical Product and Process Modeling, 4(2).

Manenti, F., G. Buzzi-Ferraris. (2009). Criteria for Outliers Detection in Nonlinear Regression

Problems. In J. Jezowski & J. Thullie (Eds.), Computer Aided Chemical Engineering (Vol. 26,

pp. 913-917).

Manenti, F., I. Dones, G. Buzzi-Ferraris, H.A. Preisig. (2009). Efficient Numerical Solver of

Partially Structured Differential and Algebraic Equation Systems. Industrial & Engineering

Chemistry Research, 48(22), 9979-9984.

Manenti, F., M. Rovaglio. (2008). Integrated multilevel optimization in large-scale polyethylene

terephthalate plants. Industrial and Engineering Chemistry Research, 47(1), 92-104.

Nelder, J.A., R. Mead. (1965). A simplex method for function minimization. Computer Journal, 7,

308-313.

Rosenbrock, H.H. (1960). An Automatic Method for Finding the Greater or Least Value of a

Function. the Computer Journal, 3, 175-184.

White, D.C. (2001). Increased Refinery Productivity through Online Performance Monitoring.

Emerson Process Management MCD Technology Division, NPRA Computer Conference.

## Comments 0

Log in to post a comment