EVOLUTIONARY MODEL G
FOR ERP PERFORMANCE
, Stefanie Leimeister
Research Institute a
t Technische Universit
rasse 25, 80805 M
Stephan Gradl, Manuel Mayer, Helmut
Chair for Information Systems, Technis
che Universitaet Muenchen
The performance of ERP systems is a critical success factor for the reliable operation of a business. A promising
approach to cope wi
th the complexity of nowadays' ERP systems and to predict their actual behavior is simulation.
Commercial ERP systems, however, only provide limited insight and thus several components have to be handled as
black boxes and require a modeling approach. In t
his paper we depict an approach to increase the accuracy of ERP
system performance simulation by using an evolutionary algorithm for modeling the black boxes performance behavior.
We can show that
are able to
generate performance mo
dels for ERP components based on
measured performance data
that describe the performance behavior of these components accurately. Furthermore we
out the characteristics of the
, as well as its advantages and disadvantages, and give an outloo
k about the
The performance of an
enterprise resource planning
system is a business critical non
, and strongly dependent on the infrastructure it is hosted on. Bögelsack et al.
showed that changes in the infrastructure c
an significantly influence the performance and
consequentially the usability of an ERP system.
Performance predictability for ERP systems is thus very
as it allows the handling of performance problems before they occur
(Balsamo et al., 2004)
reducing the risk of infrastructure changes significantly.
At the same time performance prediction i
hard to achieve
due to the complexity of modern ERP
(Anderson and Mißbach, 2005)
Often this complexity is
yzing the internal structure
of the ERP system (white box approach), identifying the correlation between the internal components, and
performing performance prediction by simulation.
Simulation though brings some “smallest elements”,
components that cannot
be seen inside (e.g.
and that have to be
The performance behavior of these black boxes is often modeled using simple mathematical
functions like the mean value on measured performance data (see e.
, though ignoring
the characteristics o
f the underlying infrastructure
As Noblet et al.
state simulation results can only be valid and trusted if the simulation is as close to
reality as possible. Our aim is to optimize the simulation results by increasing the accuracy of the black box
performance models. For this, we
an approach to model the performance behavior of these black
boxes using an evolutionary algorithm
(Zitzler and Thiele, 1999)
performing a multi
Zitzler and Thiele, 1998)
on measured performance data. In contrast to exact mathematical or algori
modeling, the evolutionary approach promises usable approximations even of multidimensional models in an
acceptable time span
(Gwozdz and Szlachcic, 2009)
, providing the possibility to consider multiple factors for
e time prediction of a black box.
The approach presented in this paper is intended to enhance the interface between the ERP performance
measurement and the ERP performance simulation.
The most related work is shown in table 1
, where we also
point out which subject the documents deal with.
documents that deal with modeling as well as
simulation describe a hybrid approach, as it is developed in this paper.
concerning performance measurement, modeli
ng and simulation
Bögelsack et al.
Gradl et al.
Rolia et al
Kraft et al.
Bögelsack et al.
Structure of the Algorithm
Oriented at natural evolution the evolutionary algorithm sets up a population of concurring threads, each
trying to generate a model that best matches the given set of measured performance data of the ERP black
box component. A s
election of the fittest is done by competition, and evolution evolves by constantly passing
better models to threads that lost competitions, and by mutation of these models.
Competition in this context
means that the fitness values of two threads (defined
by the fitness function described later) are compared,
resulting in the thread with the better fitness value to win the competition.
In order to create the performance model, several items have to be introduced. First, a central component is
created at startup. This component m
anages the population of threads that generate the performance model by
creating them at startup and by selecting an available opponent whenever a thread is in time to compete
Schematic representation of the functionality of the evolutionary algorithm.
against another thread.
Furthermore, the central component monitors the results
of each competition, to check
if the end criterion is reached.
The population size
is restricted by the
the algorithm is executed on
. A higher
population will result in higher parallelization of the evolution.
Figure 1 is a schematic sketc
h of the
functionality of the algorithm.
Each thread stores a representation of its actual model in memory. This model is represented by an object tree
that has method objects as nodes and fixed value or variable objects
Method objects represent a mathematical operator.
Mathematical operators can be binary operators like
addition, subtraction, multiplication, division and power
also unary operators like sine
are nodes in the tree, as they require param
eters, either again method objects, or fixed value or variable
objects. Fixed value objects are leafs that are set to a fixed numeric value. Variable objects are
set to the parameter values of the measured performance data used as basis for
the modeling on evaluation
time. They represent the variables in the model.
the representation of a
to illustrate the
of the model as an object tree
are method objects, where
are leafs and variable objects, and
is also a leaf
and a fixed value object
represents a number in this example)
During the competition the models of the two opponent threads are evaluated. For every set of available
performance data the performance parameters (like number of
request) are set
for the variable objects, and the modeled response time is calculated.
Based on the relative deviation for every given performance data entry, an error index
is calculated as
the sum of the deviation
every available measured
the corresponding value
the mathematical model
, divided by the number of available performance data entries
implemented directive to achieve
In this form
is the number of available performance data
the measured response time
for data entry i, and
the modeled response time for that data entry.
The thread with a lower error
wins the competition and passes its mode
l to the loser thread.
We chose this fitness function, as it describes the distance between the model an
d the measured values
, and it is eff
The negative side of the function is that it allows big deviations in
some points, w
hen others are modeled very close to the measured values. The evaluation of other fitness
functions will be part of the algorithm optimization process.
Example of the representation of a mathematical model as an object tree.
Model Passing and Mutation
Beside the competition, inheritance is the second important factor of an
evolution. After a fitter thread has
been identified by competition, it has to pass its model to the loser thread in a way that the chance for
keeping the positive characteristics of the model is high, but at the same time there is a considerable chance
or optimizing the model by mutation.
Passing a model from the winner thread to the loser thread is done by deep cloning of the object tree
representing the model, and replacing the model stored by the loser.
After the model is passed, the loser
lf mutates its new model. In
our approach with a given
either a fixed value object,
or a method object
for mutation. If a fixed value object is selected, a random value
is added to its value. The selection of a variable o
results in the allocation of
a random performance
that a method object is chosen
, the mathematical operator is set to a
randomly selected one
with the same number of parameters
In the latter case t
of the method
The optimal probabilities for mutating a fixed value,
will be analyzed by
experiments and future case studies.
By performing continuous optimization, there is a risk of getting stuck in a local opt
. We mitigate this problem of lo
cal optima by re
bearing every thread that lost 10.000 times in a
row, resulting in a re
initialization of the model of this thread. The threads themselves keep track of the
number of failed competitions. When the limit is reached, they trigger re
zation, drop the existing
model, and create a new one from scratch. Even when optimization is advanced, “rebirth” opens a way for
leaving local optima for finding the global one.
As the end criterion,
an error index limit has to be
The error index is a
indicator of a
model’s distance to the measured data, and a
in every competition, the end criterion is
checked without additional effort.
Choosing the error index as end criterion though
, as menti
involves the disadvantage of
allowing big deviations for some data entries, if the majority of the data entries are modeled very exactly.
This might lead to unacceptable prediction errors if the measured performance data is not equally distrib
including the algorithm to stop on a model that is not usable
This disadvantage though is solved
improving the fitness function.
A prototype of the evolutionary algorithm has been implemented, and applied to the measured data
as sample data
. A subset of the measured data was used for modeling, while
the rest served for validating the model.
First results show that the approach delivers usable result
s (average error less than
) when the data
used for modeling was equally distributed, while the error becomes big if the data is unbalanced. Future
improvement like weighting the input data will be necessary to remove the requirement of equally distribut
Another important factor for efficiency of the presented approach is the configuration of the evolutionary
algorithm, especially the mutation. First experiments
we conducted on the SAP benchmark data
the selection of a fixed valu
e or variable object in 95% of all mutations, and the selection of a method object
in 5% of all cases, results in a fast evolution with reliable convergence to an optimum.
For the relatively small set of input data with around 200 entries the prototype ret
urned a usable model
after around two to three minutes
when hosted on
two Intel Core2 Duo machines (1.6 and 3 GHz, both 4GB
The scalability of the evolutionary algorithm itself will have to be tested in future case studies.
l implementation of the evolutionary algorithm showed the
of the depicted
approach. Further, the first case study pointed out the efficiency, but also difficulties of the evolutionary
As next steps, further, more complex case studie
s will be performed. For this, we will develop an
interface for integrating the generated models into a LQN simulation of a SAP system
(Gradl et al.
Executing the simulation with the traditional black box modeling and afterwards using the generated models
will deliver comparable results.
In parallel, the prototype will be extended and optimized. A literature review about the vehicle routing
problem, a field where evolutionary algorithms are applied since many years, revealed the complexity of
possible configurations and modifications of the algorithm. Using the LQN simulation, we will analyze
configurations and develop an optimal a
lgorithm for the field of ERP performance simulation.
Anderson, G. W. and Mißbach, M., 2005.
Testing und Performance
SAP Press, Bonn, Germany.
Balsamo, S. et al., 2004. Model
based performance prediction in softwar
e development: A survey.
IEEE Transactions on
Vol. 30, No. 5, pp. 295
Bögelsack, A. et al. 2008. An Approach to Simulate Enterprise Resource Planning Systems.
MOLDT, D. & AUGUSTO, J. C. (eds.)
onal Workshop on Modelling, Simulation, Verification and
Validation of Enterprise Information Systems, MSVVEIS
2008, In conjunction with ICEIS 2008.
Bögelsack, A. et al. 2010. Performance Overhead of Paravirtualization on a
n Exemplary ERP System.
Conference on Enterprise Information Systems.
Funchal, Madeira, Portugal.
Gradl, S. et al., 2009. Layered Queuing Networks for Simulating Enterprise Resource Planning Systems.
D., AUGUSTO, J. C. & ULTES
NITSCHE, U., eds. 7th International Workshop on Modelling, Simulation,
Verification and Validation of Enterprise Information Systems, MSVVEIS
2009, In conjunction with ICEIS 2009,
May 2009 2009 Milan, Italy. INSTICC PRESS, pp. 85
Gwozdz, P. and Szlach
cic, E., 2009. An Adaptive Selection Evolutionary Algorithm for the Capacitated Vehicle Routing
Logistics and Industrial Informatics, 2009. LINDI 2009. 2nd International, 10
12 Sept. 2009 2009. pp.
Jehle, H. 2009. Performance
es Portalsystems in virtualisierter Umgebung am Fallbeispiel SAP.
Workshop 2009. 3. Workshop des Centers for Very Large Business Applications (CVLBA).
K.; Krcmar, H.
Kraft, S. et al. 2009. Estimating service resource consumption from response time measurements.
Proceedings of the
Fourth International ICST Conference on Performance Evaluation Methodologies and Tools.
Pisa, Italy: ICST
(Institute for Computer Sciences, S
Informatics and Telecommunications Engineering).
Noblet, C. M. H. et al., 2004. Enabling UMTS end
end performance analysis.
3G Mobile Communication
Technologies, 2004. 3G 2004. Fifth IEE International Conference on, 2004 2004. pp. 29
na, S. et al., 2008. Hybrid Performance Modeling and Prediction of Large
Scale Computing Systems.
Intelligent and Software Intensive Systems, 2008. CISIS 2008. International Conference on, 4
7 March 2008 2008.
Rocha, M. and Neves
, J. 1999. Preventing premature convergence to local optima in genetic algorithms via random
Proceedings of the 12th international conference on Industrial and engineering applications of
artificial intelligence and expert systems: mu
ltiple approaches to intelligent systems.
Cairo, Eygpt: Springer
New York, Inc.
Rolia, J. et al. 2009. Predictive modelling of SAP ERP applications: challenges and solutions.
Proceedings of the Fourth
International ICST Conference on Performance
Evaluation Methodologies and Tools.
Pisa, Italy: ICST (Institute for
Computer Sciences, Social
Informatics and Telecommunications Engineering).
Schneider, T., 2006.
SAP Performance Optimization Guide.
Galileo Press, Bonn, Boston.
Woodside, M. 2002. Tutoria
l Introduction to Layered Modeling of Software Performance. Available:
Zitzler, E. and Thiele, L. 1998. Multiobjective Optimization Using Evolutionary Algorithms
A Comparative Case
EIBEN, A. E., BÄCK, T., SCHOENAUER, M. & SCHWEFEL, H.
Parallel Problem Solving
PPSN V, 5th International Conference, Amsterdam, The Netherlands, September 27
Zitzler, E. and Thiele, L., 1999.
Multiobjective Evolutionary Algorithms: A Comparative Case Study and the Strength
IEEE Transactions on Evolutionary Computation,
Vol. 3, No. 4, pp. 257