Semi-Automatic Merging of Two Reference Model Versions: The BP3M Approach

inexpensivedetailedNetworking and Communications

Oct 23, 2013 (3 years and 11 months ago)

85 views



Eindhoven University of Technology
Department of Industrial Engineering and Innovation Sciences




Supervisor TU/e
dr. ir. R.M. Dijkman

Supervisors Pallas Athena
Maria Zimakova
Dolf Grünbauer

Eindhoven, June 14, 2011





Master’s Thesis


Semi-Automatic Merging of Two
Reference Model Versions:
The BP3M Approach

By
Emmy Dudok



i

Abstract
In business process management, reference models are provided for companies as a solid and
sometimes mandatory basis for a company’s operational business processes. A company may
adapt the reference model to create an operational model. Examples of reference models are
the standard Dutch municipality processes created by the NVVB (i.e. “Nederlandse Vereniging
Voor Burgerzaken” / “Dutch Association of Civil Services”), which every municipality adapts
toward their own specific needs (e.g. Amsterdam’s marriage license request process might be
different from Utrecht’s). Reference models may be updated over time (e.g. due to a change in
regulations) and the operational processes that were derived from it, have to be updated as
well. Hence, changes made to the ‘old’ reference model have to be obtained and applied to the
‘new’ reference model. As this procedure is currently performed manually and requires a lot of
resource capacity, there is a need for a procedure that supports the users, relieves resources
and preferably shortens the execution time of the procedure. Current procedures are
incomplete, often have unrealistic assumptions, or are not completely automated. In this
thesis, a complete semi-automatic approach is described which is able to merge two versions of
a reference model and accounts for user preferences. The developed procedure fulfills all
requirements, is highly flexible with respect to user preferences, and its intermediate results
(e.g. model mappings, change logs) may be used for other purposes (e.g. using the mapping to
determine the similarity of two business processes to consider the possibility of a merge).
Performance results of the approach’s components are provided and a running example is used
to show the intermediate and final results of the approach.

iii

Preface
This Master's Thesis serves as a closure for the Business Information Systems master program
at the Eindhoven University of Technology. The research has been carried out for Pallas
Athena in Apeldoorn, a market leader in Business Process Management software, solutions and
services. The company, founded in 1993, accounts for over 1800 customers and over one
million users around the world. It is a large, highly innovative company which offers a state-of-
the-art product. A few years ago, they initiated research on automating the procedure of
merging multiple versions of reference models. This resulted in the ability to view differences
per object between model versions based on unique identifiers (Belgers, 2005). However, as
identifiers of objects as well as the content of an object might change, a comparison solely
based on identifiers is incomplete as some similarities and differences between the model
versions cannot be identified. Additionally, the customer still has to apply the changes
manually, which requires a lot of resource capacity. Currently, merging business processes is an
active research field within the business process management branch of trade as it promises
many benefits for users of these systems, such as relieving resources caught up in the
procedure, decreasing throughput time of the merging procedure, guaranteed quality, etc.
Business processes are everywhere to be found, and this research area is increasingly getting
more attention as companies see and experience the benefits of proper business process
management. I am really interested in this highly promising research area, with a keen interest
in the practical application of the research. I therefore really appreciated the effort my
supervisor dr.ir. Remco Dijkman put into acquiring an assignment on this edge of research and
practice. In this respect, the BIS master equipped me with a solid base of theory to take up
this challenge.
I especially want to thank both of my supervisors dr.ir Remco Dijkman and Maria
Zimakova who supported and guided me throughout the entire project. In addition I would
like to thank Dolf Grünbauer, Maarte van Hattem, and Paul Eertink of Pallas Athena for their
support in the project. I thank all employees at Pallas Athena for giving me a great time while
I was there. Last but not least, I would like to thank my friends and family for their support
throughout the entire process.

Emmy Dudok


v

Table of contents
1 Introduction .................................................................................................................. 1
1.1 Motivation ........................................................................................................... 1
1.2 Goal ..................................................................................................................... 4
1.3 Approach ............................................................................................................. 4
1.4 Scope ................................................................................................................... 6
1.5 Blueprint models ................................................................................................. 6
1.6 Outline of report ................................................................................................. 7


2 Literature review ........................................................................................................... 9
2.1 Two types of merging .......................................................................................... 9
2.2 Merging requisites ............................................................................................. 10
2.3 Design criteria ................................................................................................... 13
2.4 Overview existing approaches ........................................................................... 14
2.5 Limitations of the available research. ............................................................... 14
2.6 Conclusion ......................................................................................................... 15


3 Generic metamodel representation .............................................................................. 17


4 Model mapping .......................................................................................................... 19
4.1 ICoP framework ................................................................................................ 19
4.2 Searchers ........................................................................................................... 20
4.2.1 Similar label wrapup searcher ........................................................................ 21
4.2.2 Similar description searcher ........................................................................... 22
4.3 Boosters ............................................................................................................. 23
4.3.1 Cardinality booster ........................................................................................ 23
4.3.2 Subsumption booster ..................................................................................... 23
4.3.3 Graph distance booster .................................................................................. 23
4.4 Selectors ............................................................................................................ 24
4.4.1 Match similarity selector ............................................................................... 24
4.4.2 Mapping similarity selector ............................................................................ 24
4.5 Evaluators ......................................................................................................... 24
4.5.1 Graph edit distance evaluator ....................................................................... 24
4.5.2 Path relation evaluator .................................................................................. 24
4.6 Parameters ........................................................................................................ 24
4.7 Running example ............................................................................................... 25


vi
5 Evaluation model mapping ......................................................................................... 27
5.1 Evaluation set-up .............................................................................................. 27
5.2 Evaluation results.............................................................................................. 30
5.2.1 Searchers ........................................................................................................ 30
5.2.2 Boosters ......................................................................................................... 32
5.2.3 Selectors and evaluators ................................................................................ 33
5.3 Conclusion ......................................................................................................... 37


6 Log manufacturing ...................................................................................................... 39
6.1 Creation of change logs ..................................................................................... 39
6.2 Dependent change operations ............................................................................ 44
6.3 Running example ............................................................................................... 45
6.4 Conclusion ......................................................................................................... 46
6.5 Manufacturing evaluation ................................................................................. 47


7 Log merging................................................................................................................. 49
7.1 Conflicting change operations ........................................................................... 49
7.2 Conflict resolution scenarios .............................................................................. 50
7.2.1 Scenario 1: Customer-based ........................................................................... 51
7.2.2 Scenario 2: Reference model-based ................................................................ 55
7.2.3 Scenario 3: Hybrid approach .......................................................................... 57
7.3 Conclusion ......................................................................................................... 59
7.4 Evaluation Review ............................................................................................ 59


8 BP3M Evaluation ........................................................................................................ 61
8.1 Quantitative evaluation .................................................................................... 61
8.2 Qualitative evaluation ....................................................................................... 61


9 Blueprint Models ......................................................................................................... 63
9.1 Solvable issues ................................................................................................... 63
9.2 User evaluation.................................................................................................. 63


10 Conclusion ................................................................................................................... 65
10.1 Research questions revisited .............................................................................. 65
10.2 Properties of Implementation ............................................................................ 66
10.3 Future work....................................................................................................... 67


References ........................................................................................................................... 69
Appendix A Reference model objects ................................................................................. 73
Appendix B Performance mapping .................................................................................... 76
vii

Appendix C Compound change operation ......................................................................... 77
Appendix D Considered attributes .................................................................................... 78




1
Introduction
In this chapter an i
ntroduction is provided for the problem
1.1), objective (section 1.2)
, approach
discussed.
Additionally, in section 1.5
outline of the report is
provided.
1.1 Motivation
Within business process management,
“conceptual models
that formalize recommended practices for a certain domain” (p.3,
Rosemann and van der Aalst,
(p.5, Eertink, 2010) and
are usually ad
many other process models, reference models are regularly updated
update the user’s version of the model.
user with an overvi
ew of the changes made to a model, i.e. they describe how to get from one
model to another.
Let us discuss a general example to clarify the business case.
In Figure 1.21
all steps and model versions
creates reference models
for Dutch municipalities
reference models are provided to the NVVB’s users (i.e. municipalities
Rotterdam, etc.), whom
adapt the models toward their needs
model. When
the government decides to change certain regulations
model, the NVVB has to
updat
model is again provided
to the
changes made to the previous model
the new reference model (4).

The business case considers a reference model
considers an updated
new reference model


1
The NVVB or the
“Nederlandse Vereniging Voor Burgerzaken” (the Dutch Association
2
from the original reference model (as described above). However, other scenarios are equally
applicable for the business case. For example, when there are two subsidiaries in a company,
which have created their business processes from the exact same process model and eventually
want to merge their business processes. In Box 1.1 an elaborate running example is provided
which is used throughout the research. It clarifies the merging problem.
Running Example.
This example provides an initial idea of the difficulties faced when merging
two processes. The NVVB might for example create an initial reference model of a simple marriage
license request (shown in figure 1.2a). In this model, regulations and standard practices are
captured. Local council X adapts the model to fit its own needs (shown in figure 1.2b). The red
dashed boxes indicate the differences with the original model. Within local council X it is also
possible to initiate the procedure by filling out an online form
; the application is restricted to
applicants who are both inhabitants
of the district; the ‘check identity’ is refined
into two separate
steps; the applicants receive a notification
when the request is denied; the marriage certificate is
created two days
before the wedding; and the applicants receive a notification
of their marriage
license. A year later, regulations and standard practices are updated, which results in the NVVB
creating a new version of the model (see figure 1.2c). In the new reference model there is no
inhabitant restriction
anymore and an administrative part is added
on request of many users of the
reference model. Now, local council X has to merge these two versions (i.e. 1.2b and 1.2c) into a new
operational model. For example a model such as figure 1.2d could result.
If we would take the ‘plain’ union of the old customer-specific model and the new basic reference
model we observe that there would be several inconsistencies. For example, the customer has
changed the requirement for district inhabitants. However, that requirement is no longer present in
the new reference model. Should the requirement of the user be applied to the new model, or not?
Additionally, similarities and differences might remain undetected and result in redundant elements
in the merged model. For example, the customer has refined the ‘identity check’ into two tasks, but
without a proper equivalence check we might not know these tasks represent the same unit of work
(Dijkman, 2007) and they might all be inserted into the merged model, resulting in redundant
behavior. This also holds for the additional administrative part where only considering identifiers
would result in different identifiers and redundancy in the merged model. Furthermore, it is left to
the customer (in case it is not a fixed regulation) to decide upon creating the certificate 1 or 2 days
upfront. Additionally, by only considering the two version models and disregarding the original
reference model, vital information is lost about the source of the changes. For example, we would
not be able to tell which elements the customer actually changed without considering the original
reference model.


3















































Figure 1.2: Running Example Reference Model Versions. (a) V1: Original Reference Model. (b) V1.1: Original User Model.
(c) V2: New Reference Model. (d) V2.1: New User Model.
Box 1.1 Example Merging Problem
(
d
)

(
c
)

(
b
)

(
a
)

4
1.2 Goal
At the moment,
the customer has to manually detect the changes made to the previous
reference model and apply them to the new reference model. Manually merging the models
brings about a high workload on a business’ resources
manual merging procedure
might be high
performing a manual merge, there is always a risk of accidently neglecting some aspects of a
model due to human error.

welcome. As
Pallas Athena wants to s
required resource capacity,
support version management
possible,
the following general research question was determined:
How can we support
the process of merging two reference model versions?
1.3 Approach
As a first step, literature will be studied to gain insight in existing approaches for (process)
model merging (see Chapter 2). Based on
solve the merging problem.
In
so-called manufacturing
regards the creation of
the identification of dependencies among the change operations contained within the change
log. After their creation, b
oth change logs are
￿
(V1,V2.1)) which can be applied to the original
the new user model).
This is called
and resolving conflicts.
Additionally,
the change logs. A mappin
g indicates
the models (
Weidlich et al., 2010
problem at hand which we
call
To
gether, these elements form the BP3M approach, or the Business Process Mapping,
Manufacturing and Merging approach.
In Figure 1.4 the
relations
required inputs and outputs
of each element is indicated.
process models all three
possible
New Reference Model
Original Reference Model
V2
V1
￿(V1,V2)
￿(V1,V1.1)
￿(V1,V2.1)
Figure 1.3 Approach base
d on change logs
the customer has to manually detect the changes made to the previous
reference model and apply them to the new reference model. Manually merging the models
brings about a high workload on a business’ resources
. Also,
the throughput time of the entire
might be high
a
nd as Gottschalk et al. (2008) indicated, when
performing a manual merge, there is always a risk of accidently neglecting some aspects of a

Therefore, some support for the merging
procedure
Pallas Athena wants to s
upport the end-
users of their product by
support version management

and reduce throughput time wherever
the following general research question was determined:

the process of merging two reference model versions?
As a first step, literature will be studied to gain insight in existing approaches for (process)
model merging (see Chapter 2). Based on
this literature study we
determined
In
Figure 1.3
, the idea behind the approach is provided.
The approach to merge two models is
based on combining the changes both versions
made to the original (reference)
these
changes have to be determined and are
captured in a so-
called change log
change log ￿
(A,B) between two models A and
B
captures the required changes that have to
be applied to model A
to obtain model
our case two change logs
capture the changes made by both versions
with respect to the original model (i.e.
￿
(V1,V1.1) for the original user model and
￿
(V1,V2) for the new reference model).
regards the creation of
a change log from on
e model to another and
the identification of dependencies among the change operations contained within the change
oth change logs are
then
combined into a merged change log (i.e.
(V1,V2.1)) which can be applied to the original
model to obtain the merged model V2.1
This is called
merging
and includes integrating two models by detecting
Additionally,
we need to map
the models in order to
g indicates
the (best) matches of activities (or activity sets) between
Weidlich et al., 2010
). In total, three elements are required
to solve the merging
call
(model) mapping, (log) manufacturing
, and
gether, these elements form the BP3M approach, or the Business Process Mapping,
Manufacturing and Merging approach.

relations

between the elements of the BP3M approach
of each element is indicated.
First of all, based on the available
possible
mappings are created between the original reference model
5

the new reference model and the original user model. Using the two mappings between the
original reference model and its two adapted versions, and the three process models, change
logs are created between the original reference model and its two adapted versions including
dependencies within the change logs. Using the mapping between the original user model and
the new reference model and both change logs, any conflicts between the two change logs are
indicated. A black box is used in the merging phase to resolve the conflicts by prescribing
what to do in case of a conflict. The black box captures the user preferences and results in
specific conflict resolution scenarios: user-based scenario (user changes are always preferred),
reference model-based scenario (reference model changes are always preferred), hybrid
scenarios (depending on the rules defined in the black box, either model is preferred). The
result is a merged change log which can be applied on the original reference model to obtain
the new user model.


We can now define three specific research questions for the complete problem:
- How can we automatically determine a mapping between two models?
- How can we automatically manufacture a change log of two models (given both
models and a mapping between them)?
- How can we (semi-)automatically merge two versions of a reference model (given a
mapping of both models, both change logs from the original reference model, and the
original reference model)?
A similar approach is used to answer all research questions. We start by reviewing available
literature on the topic within the business process modeling research area and across other
research areas. Then, an algorithm is created which fits the requirements of the business case
and answers the related research question. For every algorithm, the running example (see Box
1.1) is used to indicate the outcome of the algorithm for the example. Finally, an evaluation is
performed to test the performance of the algorithm.
Figure 1.4 Overview of relations between the elements of the BP3M approach

6
1.4 Scope
The algorithms of the three elements of the approach can be extended with additional
(possibly better performing) components. As time is limited for this research, and the focus is
on actually merging the model versions, all phases are researched and implemented to the
extent that is required for a solid proof of concept.
Within BPMone, four reference model types are distinguished: process models, libraries (for
efficient use and reuse of model parts), organization models (describe authorization and
distribution of work) and indicator models (to control compliance with regulations, “standards,
performance indicators, objectives and risks” (p.5, Protos Indicator 4.1 User Manual, 2007)).
All types are subject to the version management problem at hand and as they can be
connected to each other, connected elements should be taken into account when merging two
versions as the accompanying models should also be merged. As time is limited, the
implementation only considers unconnected elements of business process models. Connected
elements and other model types are left as future work.
Unfortunately, an extensive evaluation on the entire approach was not feasible within the
scope of the research due to a lack of available resources (not enough real-life model sets) and
time limitations. As such, this is left as future work.
1.5 Blueprint models
The originator of the reference model (e.g. the NVVB) could decide that some parts of the
model may not be adapted by the customer (e.g. in case of a fixed regulation). These parts of
the model are called blueprinted parts. It is for example possible that the high-level process
structure should be fixed (read-only) but the lower-level parts are adaptable by the user
(Eertink, 2010). This is comparable to when the exterior and layout of a house is fixed, but the
interior is left to your own taste. In the blueprint process model in Figure 1.5a the three
elements are fixed but the interpretation of the content of the elements is left to the user. In
Figure 1.5b we see an invalid adaptation of the model, as the ‘check identity’ element is
removed while this is restricted. In Figure 1.5c we see a valid adaptation of the model, in
which the ‘check identity’ is given form.



Check
identity

Approve
license

Request
license

Obtain
identification

Check ID-
card

Check
alternative ID

Check
identity

Approve
license

Request
license

Approve
license

Request
license

(a)
(c)

(
b)

Figure 1.5 Example blueprint model. (a) Blueprint model. (b) Invalid adaptation. (c) Valid adaptation.

7

It is also possible that some specific elements of the model may not be adapted (e.g. cannot
be relocated, properties of the elements cannot be changed, etc.) Hence, the difference between
a blueprint (part) and a reference model is the extent of static behavior.
From the above discussion on blueprint and reference models, we see that blueprinted parts
might be able to partially provide a solution to the merging problem. As blueprint models
cannot be changed by users, whenever these blueprint models are updated it is guaranteed
that no conflicts arise (this does not hold for newly added blueprint parts, as users could have
added equivalent parts which might pose conflicts). However, the question remains to what
extent blueprint models are feasible to be implemented. Hence, it should be investigated to
what extent reference models are currently left unchanged by the customers and for what
purposes blueprinted parts are required (e.g. implementation of a fixed regulation) in order to
determine the feasibility of blueprint models. Also, it can be investigated to what merging
difficulties blueprint models provide a solution. The research question determined for this
separate part of the research is:
To what extent are blueprint models able to solve the automatic merging problem?
The approach to solve this research question is to set-up a customer evaluation to check the
extent of user adaptation of the reference models. Additionally, during the main research it
could be considered for which difficulties the blueprints could provide a solution. As this can
be regarded as a completely separate research and infeasible within the scope of the research,
only a brief discussion will be provided on the topic.
1.6 Outline of report
In Chapter 2, the complete literature review is provided. Chapter 3 introduces the
metamodel which is used throughout the research. The successive chapters follow the problem
description as described earlier. See also Figure 1.6 for an overview of the chapters: Chapters 4
and 5 describe the algorithm and evaluation of the mapping phase respectively; Chapter 6 the
algorithm and evaluation of the manufacturing phase; and Chapter 7 the algorithm and
evaluation of the merging phase. Chapter 8 describes the set-up of an extensive evaluation on
the entire procedure. Chapter 9 provides a brief discussion on blueprint models. Finally,
Chapter 10 provides a conclusion and discussion on the research and indications for future
work.



Figure 1.6 Overview of chapters linked to the BP3M approach

8



9

2 Literature review
This chapter provides a literature review on the area of business process model merging. It will
provide a guideline for our research as well as information on and the extent of available
research.

As a guideline for this literature review we use some of the aspects indicated in the review
of Mens (2002) on software merging and tools. The research area of software merging is a
highly related and more mature research area and many of its topics are applicable and found
in business process merging. In both research areas parallel development results in different
versions that are controlled by version management or control: deals “with the evolution of a
software product into many parallel versions and variants that need to be kept consistent and
from which new versions may be derived via software merging" (p.449, Mens, 2002). The most
important aspect of Mens (2002) which is applicable for business process merging is the
distinction between two-way and three-way merging and is discussed in section 2.1.
Additionally, we find that there are several merging requisites required to resolve the problem
at hand which are discussed in section 2.2. Mens (2002) also indicated some design criteria of
implementations which are discussed in section 2.3. In section 2.4 an overview will be provided
which considers available research on the discussed aspects of business process merging.
Furthermore, limitations of available literature are discussed in section 2.5 and a conclusion is
provided in section 2.6.
2.1 Two types of merging
In software merging, two types of merging known as two-way and three-way merging are
distinguished. They resemble company-merging and version management merging in business
process merging respectively. See Figure 2. for a visualization of the two types. The difference
between the two is that two-way merging (2.1a) does not rely on a common ancestor, whereas
three-way merging (2.1b) uses the information obtained from a common ancestor (Mens,
2002).

a) b)
V1.2

V1

V1.1

V1.3

V1

V2

V3

Figure 2.1: Visualization Types of Merging. a) Two-Way Merging. b) Three-Way Merging.

10

Other than version management, in case of a company merger there usually is no common
ancestor available. Here, some available approaches on business process merging are discussed
for both types of merging.
Two-way merging. Both Sun et al. (2006) as well as Mendling and Simon (2006) approach
the merging problem by first combining the models in one model encompassing the behavior of
both models, after which they remove redundancy (i.e. resolve conflicts). La Rosa et al. (2009)
take a similar approach, but leaves the resolution of conflicts to the end user by appending
differences as branches. The approach taken by Gottschalk et al. (2008) is specific for the
Event-driven Process Chain (EPC) modeling language and is meant to merge two independent
processes. Their approach consists of four steps: 1) map the two models to determine the
equivalent functions (activities); 2) transform the EPCs into function graphs which only
describe the behavior of a model by its functions and their interaction; 3) merge function
graphs; 4) convert function graphs back into EPCs again. Their resulting model may have
additional behavior which was not present in either of the input models.
Three-way merging. One of the three-way merging approaches for process models is
suggested by Küster et al. (2008a) and Küster et al. (2008b). The versioning issues they
discuss stem from Business-Driven Development in which “business process models are
iteratively refined, from high-level business process models into models that can be directly
executed” (p.1, Küster et al., 2008b). Their approach consists of three steps: 1) determine
differences between two process models based on correspondences; 2) visualize differences; 3)
resolve differences by defining change operations which can be applied to the ancestor to
obtain a merged version. In Küster et al. (2009) conflicts and dependencies within and between
change logs of the ancestor to both versions are detected based on a mapping and are resolved
to obtain a merged change log. Business process version management also has a text-based
equivalent. Text-based differencing and merging considers the storage of differences between
two versions of a text file which can then be applied to a new text file. However, as Könemann
(2009) indicates in his paper on model-independent differences, the problem of model
differencing and merging is somewhat more complex than the text-based version: “model
elements may have a more complex structure than strings” (p.38, Könemann, 2009). Model
elements might have properties defined and models might have cross-references which do not
occur within text files.
Both two-way and three-way merging characterize themselves by comparing the models,
determining the differences between the models and resolving the conflicts by opting for either
model or making a compromise. Although the resolution of conflicts in the approaches
described may vary, we can detect a general tendency of the requirements to merge models:
using a mapping we can detect the differences and conflicts between the models, and a merging
algorithm is required which states how to resolve the conflicts. In the next section, these three
elements will be discussed.
2.2 Merging requisites
Model mapping. Mapping considers determining the equivalence of two models. Two
research areas can be indicated within business process model mapping: indicate equivalences
and differences between two models (e.g. Dijkman et al. (2009b)), which we call model
comparison, and retrieval of similar models from a database (e.g. van Dongen et al. (2008),
11

Dijkman et al. (2009a)), which we indicate with model retrieval. Exploring all possible matches
between two models is infeasible as it would result in an exponential number of possibilities.
Therefore similarity metrics are used to overcome this problem. Dijkman et al. (2009a) and
Dijkman et al. (2009b) mention three different similarity metrics: lexical, structural, and
behavioral matching. Lexical, linguistic or label matching considers the similarity of the
attributed labels to the elements. It uses the string-edit distance with a certain cut-off level to
determine the equivalence of nodes (Dijkman et al., 2009b). Structural matching considers
equivalent behavior of graphs representing the process models and uses the graph edit distance
and element label matching to determine similarity. It creates different mappings and
determines the best mapping based on mapping scores. Behavioral matching considers the
equivalence based on the execution behavior of models and element labels. It uses distance in
document vector space which is constructed using causal footprints (Dijkman et al., 2009a). La
Rosa et al. (2009) define similarity measures for every type of element, e.g. string edit
distance, context similarity, which translate into a node and model matching score. In their
review Rahm and Bernstein (2001) consider a conceptual mapping for two-way merging which
is automatically derivable and is validated and modified afterwards. An elaborate procedure
for mapping is known as the ICoP Framework (Identification of Correspondences between
Process Models) developed by Weidlich et al. (2010). This framework supports many-to-many
mappings which are common in practice and takes into account “that equivalent activities
may be modeled at different levels of granularity, have different labels, and have different
control-flow relations to other activities” (p.2, Weidlich et al., 2010). In the experimental study
of Dijkman et al. (2009b), a greedy structural matching algorithm provided the best results.
Van Dongen et al. (2008) used lexical as well as behavioral matching in their algorithm to
determine model similarities based on causal footprints and a vector model. They used
semantic similarity (lexical) and contextual similarity (the lexical similarity of adjacent events
which are EPC-notation specific) to map objects. Lu et al. (2009) discuss the Process Variant
Repository which stores process variants and supports the retrieval of process variants based
on similarity measures between a query and the retained models. The result is a ranked list of
similar models based on structural and behavioral mapping.
Difference and conflict detection. In order to merge two models it is first of all necessary
to detect any differences and conflicts between them, which will be later on resolved. The
mapping can be used to determine the differences and conflicts between the models. There are
two different types of approaches for the detection of differences: creating a change log and
indicating or visualizing differences. Usually, the mapping is required to execute the detection.
Conflicts can be detected using a merge matrix approach, which is used for example by Küster
et al. (2009). Küster et al. (2009) consider differences between two model versions as changes,
and to determine differences and conflicts change logs from the ancestor model to both
versions can be compared.
A change log clearly indicates how two versions of a model differ from each other (Chen et
al., 2004). Elementary changes (e.g. AddNode) can usually not be considered in consolidation
as they are usually part of a coherent group of dependent or compound changes (e.g. when a
node is added, so will some edges)(Küster et al., 2009). In order to detect both dependencies
within and conflicts across change logs a matrix is constructed which describes the possible
dependencies/conflicts. In software merging the approach known as operation-based merging
(Mens, 2002) resembles conflict detection based on change logs and reviews command histories
12

(resembling change logs) to see whether evolution operations (resembling change operations)
affect the same entity and might be in conflict. Change operations contained in the change log
indicate the type of change operations followed by information on the changed element and its
location. There is no standard change log representation available though. Change logs are
often maintained whenever a model is updated, but this is not always the case. When change
logs are unavailable, they can be created afterwards (e.g. Küster et al., 2008b). In software
merging change logs are also known as deltas. There are two types of deltas: symmetric deltas
which show differences, and directed deltas which show how to get from one model to another
(Mens, 2002).
Differences between two models can also be indicated in a user-friendly way. Dijkman
(2007) provided intuitive difference categories which resulted from frequent occurrence in
practice and are based on the mapping. Three main difference categories are distinguished
within which specific differences reside (e.g. different collection of roles, refined activity,
iterative vs. once-off occurrence, etc.): activity authorization, correspondence between
activities, and control flow differences. Both single elements as well as sets of elements are
addressed within the categories.
Differences might be considered conflicting or non-conflicting depending on the available
information, i.e. having the ancestor available or not. For three-way merging non-conflicting
differences exist when the difference stems from one of the models making a change to the
ancestor model, and conflicting differences exist when a difference is due to both models having
made changes to the ancestor model. For two-way merging, all detected differences are
regarded as stemming from both models having made changes.
Difference and conflict resolution. In order to resolve the conflicts, it has to be known
what the user preferences are. On a high level, different approaches are possible to merge two
models and thereby resolving any conflicts. Sun et al. (2006) distinguish dimensions which
indicate the possible type of merges between parts of the models based on objects:
- Lossless merge: all activities of both models are preserved;
- Lossy merge: not all activities of both models are preserved;
- By merging pattern: sequential (replacement and insertion), parallel, conditional,
iterative, or complex.
Regarding two-way merging, one model might be taken as a basis to which the other model is
applied (La Rosa et al., 2009 and Sun et al., 2006). Attention has to be paid to which model to
take as a basis as it was identified by Sun et al. (2006) that there is a difference in the
resulting models.
Conflict resolution is highly related to user preferences and therefore difficult to automate.
The user has to decide upon the desired dimension of merging. If the user is highly involved in
the resolution of conflicts (e.g. interactive or manual resolution (Mens, 2002)), a proper
visualization is required. For example Küster et al. (2008b) propose constructing a change log
to provide the user with an overview of the differences and enabling the user to choose from
compound change operations. As elementary change operations (e.g. Add Element, Delete
Element) are not suited for grasping higher level intuitive differences, compound change
operations (e.g. Insert Action) are used which are created in a bottom-up fashion from
elementary change operations (Küster et al., 2008b).
The overview of differences often requires dividing the model into coherent parts. Similar to
Küster et al. (2008b), Vanhatalo et al. (2007) divide the model by defining Single-Entry-
13

Single-Exit (or SESE) fragments. Zerguini (2004) indicates subsets in workflow nets or
Multiple-Entry-Multiple-Exit (MEME) fragments by using free-choice workflows (i.e. the
output set of any set of two places is either identical or has no overlap at all). A completely
different approach is taken by Hauser et al. (2007) which use region trees to determine regions
in a model based on semantically sound workflow models (i.e. all possible executions terminate
successfully, there is only one start and end node and every node is on a path from the start
node to the end node). Their approach allows for both SESE as well as MEME fragments. It
starts by creating a basic partition of the nodes which are combined into higher regions by
applying region-growing rules. All these approaches serve a structural (e.g. checking for
correctness) instead of a qualitative (proper overview for the user) purpose which results in
non-intuitive coherent model fragments. As discussed before, Dijkman (2007) provided a top-
down categorization of differences which is highly intuitive for users.
The approaches may result in merged models which provide different functionality. The
resulting model of La Rosa et al. (2009) is a so-called configurable process model (see also
Gottschalk (2009) and Rosemann and van der Aalst (2007)), in which different options are
modeled as configurations so that the customer can adapt the model according to his/her
needs. It contains all possible behavior of the models and is regarded an annotated process
graph; a process model which contains additional metadata indicating the originating nodes
and edges enabling it to extract both versions from the merged model again. The approach of
Sun et al. (2006) actually integrates the models. In their merged model it is only possible to
have elements that are in either or both of the two models. This is also the case for Mendling
and Simon (2006) albeit a merge of process views. The resulting model of Gottschalk et al.
(2008) provides a choice between different behavior but does not actually synchronize the
behavior of the models. Hence, this is applicable when there is no overlapping behavior (e.g.
merging different processes).

It is suggested to keep the approaches for conflict detection and resolution strictly
separated, in order for the techniques to be tuned without influencing each other (Mens, 2002).
2.3 Design criteria
According to Mens (2002) there are several design criteria that have to be considered when
creating a merging tool: degree of formality, accuracy, domain independence, level of
granularity, scalability, and level of automation. The higher the degree of formality of the
algorithm, the lower the practical use. It is important to detect as many conflicts as possible or
the result may be an incorrect merged model. When the accuracy is low, conflicts might be
detected without being actual conflicts and other conflicts might remain undetected. The
approach preferably has to be domain independent as it will then be uniformly applicable.
However, this is usually at the expense of accuracy as domain-independent information cannot
be used to resolve conflicts. Approaches also differ in their granularity level or level of detail of
the conflict detection. Preferably, all levels of detail are dealt with. An important aspect of the
approach is whether or not it is scalable. This indicates whether it can cope with an increase in
the size of the process models (e.g. if the execution time is polynomial or exponential with
respect to the size of the models it is not scalable (Mens, 2002)). There are three levels of
automation: manually, interactive, and fully automatic. While conflict detection is usually fully
14

automatic, conflict resolution usually is interactive or even manual. Also two-way merging
seems harder to automate than three-way merging as less information is available.
2.4 Overview existing approaches
In Table 2.1 an overview of the aspects of existing approaches is provided. Additionally, the
last row indicates what aspects would be desired (but may not be feasible) for the problem at
hand.
Approach

Merge
Type
Mapping
Type
Similarity
Metric
Conflict
Detection
Type
Conflict
Resolution
Type
Level of

Automation
Domain
-
Dependence
Dijkman (2007)

-
-
-
Visualize
Differences
-
-
-
Dijkman (
2009a
)

- Model
Retrieval
- - - - -
Dijkman (
2009b
)

-
Model
Comparison
Lexical,
Structural
-
-
-
-
Van Dongen et al.
(2008)
- Model
Retrieval
Lexical,
Behavioral
- - - EPC
Gottschalk et al.

(2008)
2-Way
-
-
-
Configurable
Process Model
Interactive
EPC
Küster et al. (
2008a
)

3-Way - - Change Log Choice Manual -
Küster et al. (
2008b
)

3-Way
-
-
Change Log
Choice
Manual
-
Küster et al. (2009)

3-Way - - Change Log Manual -
La Rosa et al. (200
9)

2-Way
Model
Retrieval,
Model
Comparison
Lexical
-
Configurable
Process Model
Manual
-
Lu et al. (2009)

- Model
Retrieval
Structural,
Behavioral
- - - (PVR)
Mendling and Simon
(2006)
2-Way
-
-
-
-
Automatic
-
Sun et al. (2006)

2-Way - - - Merge Automatic -
Weidlich et al. (20
10
)

-
Model
Comparison
Lexical,
Structural
-
-
-
-
Desired

3-Way Model
Comparison
Lexical,
Structural,
Behavioral
Change Log,
Visualize
Differences
Merge Interactive,
Automatic
-
Table 2.1: Overview of Existing Approaches on the Discussed Aspects
2.5 Limitations of the available research.
Most approaches considered above focus only on a single element of the entire approach to
merge two models (i.e. mapping, conflict detection or resolution) while the other elements are
left as either future work, are assumed to be available upfront or left for the user to resolve.
For example it is not uncommon for the approaches to assume (e.g. La Rosa et al., 2009) or
take for granted (e.g. Küster et al., 2008b) that an initial mapping between the models is
available. Also the actual merging is usually left to the user, although supported by a
visualization of either differences or resolution options (e.g. Küster et al., 2008b). It is often
indicated that the choice on which resolution option to choose (i.e. user preferences) is
assumed to be impossible to automate (Küster et al., 2008b). Although the available
approaches tend to reduce the required resource capacity of the user, when fully automated it
might be even further reduced.
Some other shortcomings of the approaches are that many-to-many mappings are not
always supported (e.g. Küster et al., 2008b) and often only structural differences are
considered resulting in an incomplete merged model instead of also taking into account related
information and other elements of the model.
15

An assumption that is unrealistic is that it is often assumed that nodes across model
versions with equal identifiers represent similar functionality (e.g. in Küster et al. (2008b)).
However, as users are allowed to change the content of elements, this is not a fair assumption.
Nodes with equal identification can represent different units of work (Dijkman, 2007) and
nodes which represent equal unit of work do not necessarily have equal identifications. An
assumption made in Zerguini (2004) assumes workflows are free-choice, thereby creating a
restriction on the input models. Other assumptions can easily be forced upon a model though,
such as nodes always having to be on a path from start to end and activities strictly having
one incoming and one outgoing edge (Küster et al., 2009).
2.6 Conclusion
Based on this analysis of existing approaches, we will develop our approach as follows. In
our case the common ancestor is available and we require the approach to require as little user
interaction as possible. The type of merge that best fits is a three-way merge. Although we do
not have change logs available, the approach of detecting dependencies and conflicts in change
logs and combining change logs in a merged change log of Küster et al. (2009) seems to fit our
needs best and we will use this approach as a starting point. Our approach considers all
required elements to resolve the problem at hand and includes the following elements: creation
of mappings between models; (re-)construction or manufacturing of change logs and indicating
the dependencies among the change operations within the change logs; and detection of
conflicts between two change logs and resolution of the conflicts into a merged change log
which can be applied to the ancestor model to obtain the merged model. These elements are
captured in the BP3M or Business Process Mapping, Manufacturing and Merging approach as
explained in detail in the introduction (see Figure 2.2). Extending the approach of Küster et
al. (2009) we will create an approach that is able to capture predefined user settings in order
to require even less user interaction in conflict resolution. The developed approach is domain
independent in order to be generally applicable, but its implementation is narrowed down to a
well-defined domain, namely business process models in the BPMone modeling language in
order for it to be practically applicable.


Figure 2.2: Overview BP3M Approach

16


17

3 Generic metamodel representation
The purpose of this section is to provide a generic representation of business process models.
The metamodel will be referred to, and extended throughout the research. The metamodel
ensures the general applicability of the approach and provides a correctness check for the
merged model. A model is said to be syntactically correct when it is consistent and complete
against the metamodel that it is based on (Becker et al., 2000). Note that we only require a
generic metamodel, not a generic modeling language. Every business process modeling language
has its own set of elements used to describe a process. Common elements are activities, places,
edges, but also buffers, triggers, etc. can occur. Furthermore, the languages can have
additional elements and attributes belonging to all elements. In the following, we describe the
generic business process metamodel (see Figure 3.).
Our generic representation of a business process model is a graph (G) which may contain
elements (EL). The elements can be either edges (E) or nodes (N). Elements can have
attribute value pairs (AV or properties) attached which consider an attribute (A) with a value
(V). Specific element types (τ) of a business process modeling language can be mapped to the
nodes and edges to make the metamodel specific. Note that a property can only have one
value. This is either a simple value or a complex value, in which a complex value is a
composite value of multiple complex and/or simple values.


Based on these elements, we define an attributed business process graph as follows.
Definition 1 (attributed business process graph). A labeled graph G consists of elements
(EL) and is defined as a seven-tuple (N,E,T,A,V,τ,α). Here, N, E, T, A, and V are sets of
nodes, edges, node types, attributes and attribute values, respectively. The function τ maps
elements to element types, and α maps elements to attributes with attribute values. The
following properties hold:
- E ⊆ N ä N
- τ: EL Ø T
- α: (N ∩ E) Ø ￿(AäV)
*

1

*
AV
Attribute (A)
Value (V)
1

1

*

*

Graph (G)
Edge (E)
Node (N)
1

*

-
incoming

-
outgoing

1

Element (EL)
*

1

-
from

-
to

Figure
3
.
1

Business Process Meta
-
model

18


19

4 Model mapping
In this section, automatic model mapping is researched in depth and an algorithm is developed
and described. As we have seen earlier (see also Figure 4. below), a mapping (list of best
matches) is required in order to manufacture a change log between two models and is also used
as input to resolve the conflicts and merge the change logs. A mapping shows how parts of one
model map to parts of another model. The ICoP framework as developed by Weidlich et al.
(2010) is used as a starting point to design an algorithm for model mapping and is extended
where necessary. In section 4.1 the framework is explained. The succeeding sections 4.2 – 4.5
consider the components of the framework. In section 4.6 the parameters used by the metrics
are provided. Finally, in section 4.7 the running example is used to indicate the result of the
algorithm.


4.1 ICoP framework
The ICoP framework provides reusable components to develop matchers: tools that
generate mappings (Weidlich et al., 2010). There are several reasons to use this framework.
First, Weidlich et al. (2010) distinct both elementary or one-on-one (1:1) and complex or
many-to-many matches (m:n, with |m|>1 or |n|>1). Second, it has numerous metrics already
implemented which we may use, update (for the problem at hand) and combine to obtain a
sufficiently high mapping performance. Weidlich et al. (2010) distinguish four different
components of the matcher which all have separate algorithms: searchers, boosters, selectors,
and evaluators (see Figure 4.2).

Figure 4.2 The architecture of the ICoP framework extracted from Weidlich et al., 2010.
Figure 4.1 Graphical Representation Layered Problem

20

The components of the matcher are connected as follows. Searchers extract a multiset of
possible matches between activities, which is converted into a set by boosters. Boosters may
aggregate and remove matches and adapt the matching score of the matches. Finally, a
selector chooses the best matches (without overlapping parts) based on the individual
matching scores or considers a mapping score developed by an evaluator.
The goal of the framework is to develop a mapping between the elements of two models,
more specifically between the activities of the models. A mapping between two graphs G
1
and
G
2
, called Map, has the following properties:
- G
1
= (N
1
,E
1
,T
1
,A
1
,V
1

1

1
)
- G
2
= (N
2
,E
2
,T
2
,A
2
,V
2

2

2
)
- Map ⊆ (￿(N
1
) – {∅}) ä (￿(N
2
) – {∅})
A mapping between (at least) two nodes n
1
∈N
1
and n
2
∈N
2
exists when there is a tuple in the
mapping that contains both nodes. Formally this is expressed as:
- Map(n
1
,n
2
) = ∃x
1
,x
2
: x
1
⊆N
1
∧ x
2
⊆N
2
: ({n
1
} ∩ x
1
,{n
2
} ∩ x
2
) ∈ Map
In the following sections, several metrics within each component will be selected, possibly
adapted and implemented. They are then used to develop and evaluate mappings for the
problem at hand. All used metrics (algorithms) will be discussed in detail and it will be
indicated when there is a deviation from the ICoP framework. In Table 4., an overview of the
discussed metrics is provided. The succeeding sections will discuss searchers (section 4.3),
boosters (section 4.4), selectors (section 4.5), and evaluators (section 4.6) respectively.

Component Implemented Metrics
Searcher - Similar label match wrapup searcher
- Similar description match searcher
Booster - Cardinality booster
- Subsumption booster
Selector - Match similarity selector
- Mapping similarity selector
- Look ahead match similarity selector
- Look ahead mapping similarity selector
Evaluator - Graph edit distance evaluator
- Path relation evaluator
Table 4.1: Discussed Metrics per Component
4.2 Searchers
In this section, some searcher metrics are discussed. Searchers determine a first set of
matches between the models accompanied by a matching score. This is by no means a
complete list of all possible matches and can also include overlapping matches. As no
additional possible matches will be added after this phase, the outcome of this step is crucial
for the resulting performance. Searchers consider content of nodes as well as the structure of
the model (including splits and joins), but not the relation types (and/or/join) or possible
paths. There are many ways to determine whether or not two element sets are similar. The
search components discussed below are implemented for the problem at hand and can be used
to determine the initial matches.
As indicated in the introduction, and also mentioned by Weidlich et al. (2010), it cannot be
assumed that equal identifiers mean equal units of work. See for example Figure 4.3 in which
element ‘F’ of the first model has a different identifier than element ‘F’ from the second model.
21

Therefore, we need other metrics to determine equivalence. The first metric is the similar label
wrapup searcher and provides 1:1 matches. Second, as an alternative or addition to the
previous searcher, a searcher based on additional textual information is implemented. This
similar description searcher can result in both 1:1 as well as 1:n matches. It has to be noted
that due to time-restrictions structural comparisons such as the Fragment Doc Searcher
(Weidlich et al., 2010) are outside the scope of this research and are left as future work. As
such, only a lexical similarity metric is used. In Figure 4.3, the result of the two metrics is
graphically represented.


Legend

Description

Match Type



Similar Label Wrapup Searcher

1:1


Similar Description Searcher

1:1 / 1:n / n:1
Figure 4.3 Search algorithm metrics
Below, both searcher components are discussed in detail and the algorithms are provided.
4.2.1 Similar label wrapup searcher
This searcher is based on the Wrapup and Similar Label Searcher of Weidlich et al. (2010)
and identifies 1:1 matches based on the string edit similarity of activity labels in the set of
non-matched nodes. The runtime complexity only depends on the number of nodes. There are
many possible extensions to determine the similarity between two labels; for example
synonyms and homonyms could be taken into account. For this first implementation, only the
Levenshtein distance (also known as the string edit distance) is implemented (Levenshtein,
1966). This distance determines how many character operations (insertion, deletion,
substitution) are necessary to convert one string into another.
Before the distance measure is applied to strings, the strings are preprocessed by stemming
and stopword removal. Stemming a text can be regarded as normalizing words, it reduces
words to their root. This is especially useful when there are different users who created the
models or when there naming conventions do not exist. After stemming, the following words
for example would all result in ‘distract’: distraction, distractions, distractive, distracted, and
distracting. However not all words can be stemmed by only removing the ending of the word.
Consider for example the Dutch word ‘boom’ and ‘bomen’ (which mean tree and trees in
Dutch). Removing -en from "bomen" results in "bom" (which means bomb in Dutch). For
stemming to be applied properly, an algorithm is required to ensure the proper insertion and
deletion of characters to end up with the expected word. For the time being, stemming is only
implemented to the extent that some word endings are included in the stopword list for
removal. Stopword removal ensures that words which are very common in a language, and
D
E
A
B
CD
F
AB
C
E
1 2 3 4 5 1 2 3 4
Model 1 Model 2
22

may bias the ranking value, are removed. Hence, a combination of stemming and stopword
removal enhances the precision of the results of an algorithm. As there was no Dutch stopword
list in the framework yet, and the models to be tested were all in the Dutch language, one was
integrated with the implementation. The list of stopwords (and word endings) currently
accounts for 111 items
2
and can be extended to include other common biasing words.
After determining the string edit distance, it is normalized for the size of the largest string
under consideration to determine the similarity (Weidlich et al., 2010). More abstract, it is
expected that “abcd” and “abce” are equally similar to each other as “abcdabcdabcd” and
“abceabceabce”. As this measure does not account for synonyms, homonyms, order of words,
etc. identified matches will have a high confidence. The basic algorithm for the similar label
wrapup searcher is as follows:
1. Determine the string similarity using the Levenshtein distance (LD) for every
activity name (s) compared to every activity name (t) in the other model, and
normalizing for the maximum string length: ￿￿￿￿￿￿ ￿￿ ￿ ￿ ￿
￿￿￿￿￿￿￿
￿￿￿￿￿￿￿￿￿￿￿￿￿
;
2. Determine the best similarities (above certain threshold) and add these to the list of
matches.
4.2.2 Similar description searcher
As an addition to the previous searcher component, the similar description searcher is
defined which is based on the Distance Doc Searcher of Weidlich et al. (2010). This searcher
identifies both 1:1 as well as 1:n matches. The idea is to compare descriptions of activities and
groups of activities with each other to find both 1:1 as well as 1: n possible matches. To this
end ‘virtual documents’ (Weidlich et al., 2010) are determined which include all textual
information available of a single node or set of nodes (i.e. groups as indicated below). The
similarity between two virtual documents is calculated using vector space distance, with terms
as dimensions and dimension values based on term frequency. The ICoP framework considers
textual information as obtained from the activity name, authorized role name, input and
output data, and activity description. The terms in the virtual document are preprocessed
using term normalization, stopword filtering, and term-stemming. For our approach, virtual
documents include text from the node name and description. This might be extended by
including textual information of the triggers or other elements related to the activity, but this
is left as future work due to limited time. Again, the integrated Dutch stopword list is
addressed.
Nodes within both models are grouped based on their distance in the graph. This somewhat
takes into account the structure of the graph but should not be mistaken with structural
similarity metrics. Five types of groups are identified: sequences, splits, joins, subprocesses
(which is an extension to Weidlich et al. (2010)) and others. The distance doc searcher
identifies all possible groups for every node for every possible graph distance (up until a
maximum value) and removes duplicates afterwards. The runtime complexity is dependent on
the size and structure of models as well as the maximum value to which the graph distance is
set. The basic algorithm for the similar description searcher is as follows:


2
The stopwords were obtained from the following websites:
-
http://snowball.tartarus.org/algorithms/dutch/stop.txt

-
http://www.ranks.nl/stopwords/dutch.html

23

1. Obtain all possible groups (of type: sequence, split, join, subprocess and other)
within the maximum graph distance;
2. Create virtual documents for every activity and group of nodes: include name and
description of a node. Use stopword removal and stemming to preprocess the
documents;
3. Calculate the similarity between all node-group pairs based on the vector-space
similarity (or cosine similarity measure), with terms as dimensions and term
frequency used to calculate dimension values. The measure is normalized for the
length of the documents as longer documents will have larger term frequencies;
4. Select the best similarities (above threshold and without overlap) and add these to
the list of matches.
4.3 Boosters
After the searchers have identified a multiset of potential matches, boosters aggregate and
adapt the results and the matching scores. Three of the proposed boosters by Weidlich et al.
(2010) are implemented and discussed here.
4.3.1 Cardinality booster
The cardinality booster aggregates potential matches that are exactly equal (redundant
matches). It increases the matching score of the remaining match. When we have two matches
(M) between activities (A) with a matching score (s): M
1
= ({A1}, {A2}, s
1
); M
2
= ({A3},
{A4}, s
2
), and A1 = A3, and A2 = A4, then the resulting match equals ({A1}, {A2}, s
a
), with
s
a
= s
1
+ (1 – s
1
)*s
2
.
4.3.2 Subsumption booster
The subsumption booster is used to adapt matching scores for matches that subsume
another match. When we have two matches: M
1
= ({A1}, {A2}, s
1
); M
2
= ({A3}, {A4}, s
2
),
and A1 = A3, and A4 ￿ A2, then M
1
will be adapted into ({A1}, {A2}, s
a
), with s
a
= s
1
+
w
s
*(1-s
1
)*s
2
with weighting factor w
s
between 0 and 1.
4.3.3 Graph distance booster
The graph distance booster is used to adapt matching scores for matches based on the
maximum distance between activities. It determines the maximum graph distances within both
sets of matched activities, given the maximum graph distance between any two elements
within both models. Two ratios r1 and r2 are defined:
￿￿ ￿ ￿ ￿
￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿
￿￿￿￿￿￿￿￿￿￿￿

￿￿ ￿ ￿ ￿
￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿
￿￿￿￿￿￿￿￿￿￿￿

With r1, r2 ∈ [0..1]. When the average of these ratios is above a certain threshold, the
matching score is increased by:
￿￿￿￿￿￿￿￿ ￿ ￿￿ ￿￿￿￿ ￿ ￿￿ ￿ ￿￿￿ ￿ ￿￿￿ ￿￿￿￿
With w
d
between [0..1], the weighting factor. And s1 the matching score. It is also increased
by the following amount if the absolute difference between these ratios is below or equal to
another threshold:
￿￿￿￿￿￿￿￿ ￿ ￿￿ ￿￿￿￿ ￿ ￿￿ ￿ ￿￿ ￿￿￿￿￿ ￿ ￿￿￿ ￿￿￿￿￿￿
24

With w
f
between [0..1], the weighting factor.
4.4 Selectors
With the set of potential matches at hand, we can now determine a mapping. Several
mappings are constructed, keeping in mind that no included match may overlap another
included match. Unlike searchers and boosters, only a single selector can be used at the same
time. Here, we discuss two searchers.
4.4.1 Match similarity selector
The match similarity selector is the simplest selector. It bases its choice of matches solely
on the matching scores. The process of selecting matches is iterative: a match with the highest
score (above a threshold) is selected, and the set of potential matches is reduced by
overlapping matches with the current result. In case of matches having equal scores, it is either
possible to arbitrarily choose one, or it is possible to check the result of the next iteration (look
ahead).
4.4.2 Mapping similarity selector
The mapping similarity selector regards the complete mapping scores to decide on the best
mapping. Iteratively, activities are added that result in the highest score of the complete
mapping. It is possible to either arbitrarily add matches having equal scores or checking the
result of the next iteration (look ahead). It is also possible to create a selector that integrates
both the matching and mapping selectors, but these are not addressed here.
4.5 Evaluators
Finally, evaluators are used to determine the performance of a potential mapping and can
be used by selectors to base the decision for the final mapping on. Two evaluators are
discussed here.
4.5.1 Graph edit distance evaluator
The graph edit distance evaluator determines the required number of atomic graph
operations necessary to transform one graph into another using the provided mappings. An
ideal mapping is used to calculate the similarity score. According to this measure, the ideal
mapping maps all nodes and edges of a model to another model. Although an ideal mapping
for our problem might include some unmapped elements, this measurement can be used to get
an indication of the performance.
4.5.2 Path relation evaluator
The path relation evaluator considers the preservation of path relations for pairs of
matches. The more edges there are preserved, the higher the mapping score.
4.6 Parameters
In this section, the parameters which are required for each metric are provided. In Table 4.2
the parameters which are used within each metric per component are explained and the
datatype of the parameter is indicated.
25

Component Metric Parameter Datatype
Searcher



Similar label match wrapup searcher SearcherThreshold:
defines the minimum required similarity score, in
order for a match to be included.
Double
Similar description match searcher Cutoff:
defines the minimum required similarity score, in
order for a match to be included.
MaxGroupDistance:
defines the maximum distance to define groups.
DoWrapup
defines whether or not to perform a wrapup.
Double
Integer
Boolean
Booster



Cardinality booster - -
Subsumption booster BoostFactor:
defines the weighing factor to boost the matching
score, as defined above
Double
GraphDistance booster MaxDistanceBoostFactor:
defines the factor with which the matching score
is boosted, as defined above
BoostWithMaxDistanceRatioAbove:
defines the minimum threshold required to boost
the matching score
DistanceDifferenceBoostFactor:
defines the factor with which the matching score
is boosted, as defined above
BoostWithMaxDistanceRatioDifferenceBelow:
defines the maximum threshold required to boost
the matching score
Double
Double
Double
Double
Selector



Match similarity selector ThresholdSimilarity:
defines the minimum required similarity score, in
order for a match to be possibly included in the
mapping.
Double
Mapping similarity selector -
Look ahead match similarity selector ThresholdSimilarity:
defines the minimum required similarity score, in
order for a match to be possibly included in the
mapping.
Double
Look ahead mapping similarity
selector
- -
Evaluator



Graph edit distance evaluator WeightSkippedVertex:
Indicates the weight assigned to skipped or added
vertices in a model.
WeightSubstitudedVertex:
Indicates the weight assigned to substituted
vertices in a model.
WeightGroupedVertex:
Indicated the weight assigned to (un)grouped
vertices in a model.
WeightSkippedEdge:
Indicates the weight assigned to
skipped/added/substituted edges in a model.
Double
Double
Double
Double
Path relation evaluator - -
Table 4.2 Parameters per metric
4.7 Running example
Here we use part of the running example (models V1 and V1.1) to indicate the outcome of
the mapping algorithm. In the mapping between the two models is indicated. Red dotted lines
26

represent matches between node sets, whereas blue dashed lines represent matches between
triggers. Additionally, it can be seen that there is also a 1:n mapping as ‘Check identity’ is
matched to both ‘Check ID-card’ and ‘Check alternate from of ID’.



Figure 4.4 Outcome mapping algorithm for mapping between model V1 and V1.1.

Model V1
Model V1.1
27

5 Evaluation model mapping
In this section, several aspects of the mapping implementation are evaluated in order to
determine one overall mapping approach including the best performing components. First of
all, the general set-up of the evaluation is provided in section 5.1. In section 5.2 the results are
provided. Finally in section 5.3, a conclusion is given.
5.1 Evaluation set-up
According to Do et al. (2002), there are several requirements to a proper evaluation
mapping systems. We capture these elements in: evaluation approach, evaluation data, and
evaluation measurements. These elements are discussed below.
Evaluation approach. It is important that whenever possible components of a system
should be tested separately (Do et al., 2002). As there is expected to be very low cross-
component additional gain, all components will be evaluated separately. However, a higher
performance of for example searchers will boost the performance of the selectors on which they
are based (this is due to the components being related in the ICoP framework). Therefore, we
will review the component performance relatively within each component after which we will
end up with one or several overall best matchers. These matchers will again be tested. As it is
possible that combinations of searcher metrics and combinations of booster metrics achieve a
higher performance, these will also be tested. In order to test components, a complete matcher
(specification of all components) is required. For every component that is tested, all other
components will be kept equal during testing. The other component metrics will be based on
the results of that component. This results in an iterative evaluation process in which the
results of the searcher are used for the selector, and the other way around to finally obtain the
best possible results. All tests are performed on all available model comparisons and where
necessary (in case of variation in the results) repeated multiple times (5x) in order to achieve
an average with a higher confidence. Variation in the results stems from arbitrary choices the
algorithm may make.
Evaluation data. According to the guidelines of Do et al. (2002) we provide some
information on the input data used for the evaluation. The developed mapping system is
language independent, any model in any language can be mapped (also across languages) as
long as it is converted into the general directed graph used by the mapping algorithm.
However, for the purpose of evaluation a BPMone specific implementation is used. As we are
considering reference models and their customer-deviations, several reference processes are
acquired along with some adapted processes. We have acquired 3 different reference processes,
in the context of municipalities with customer-specific models of 2 municipalities (obtained
from Wagemakers (2009)). Additionally, mappings of analysts are available for these processes,
on which we can base our evaluation. On average the models consisted of 25.3 nodes (with a
minimum of 14 and a maximum of 35). A total of 116 matches (after preprocessing) were
found between the models by the analysts, of which 22 were included in a complex match. In
total there were 9 complex matches (1:n and n:m) found by the analysts. One of the important
aspects of the evaluation input is schema similarity: “the ratio between the number of
28

matching elements (identified in the manually constructed match result) and the number of all
elements from both input schemas.” (p.223, Do et al., 2002). The schema similarity for our
input equaled on average 82% which is rather high but expected for reference models (which
are derived from the original model to which they are compared). The input files had to be
preprocessed in order to satisfy the assumptions on the models (e.g. indicating and completing
all start and end indications of subprocess elements, which is domain-specific). Preprocessing
was also performed on the analysts’ mappings to enhance the quality and confidence of the
results as some indicated matches proved to be inadequate (removed) and other matches were
missing (added).
For the evaluation a NEC VERSA P550/FP550 is used with the following specifications:
- Processor: Intel® Pentium® M 750 processor (1.86GHz)
- Memory (RAM): 1.00GB
- Operating system: Microsoft Windows XP SP2
- Hard disk: 80GB
Evaluation measurements. The system is tested against a baseline mapping which is
assumed to provide the correct matches. Figure 5.1 represents how the results from the system
mapping can be compared to the baseline mapping.

Figure 5.1 Comparing baseline matches with system matches. Obtained from Do et al. (2002)
Based on these indications we can define some quality measures. The most widely used
measure is the F-score, which is based on the recall and precision. Precision represents the
fraction of system matches that is correct according to the baseline mapping and recall
represents the fraction of found system matches which are correct
￿￿￿￿￿￿￿￿￿ ￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿￿￿￿￿￿ ￿￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿

Both statistics can achieve a high performance at the expense of the other: by returning only
few correct matches (e.g. low recall), precision can be maximized; by returning all possible
matches, recall is maximized at the expense of precision. Hence, precision and recall should be
combined to assess the performance. The F-score represents a harmonization between precision
and recall:
￿ ￿￿￿￿￿￿ ￿ ￿ ￿
￿￿￿￿￿￿￿￿￿ ￿ ￿￿￿￿￿￿
￿￿￿￿￿￿￿￿￿ ￿￿￿￿￿￿￿

The above F-score is also known as the F
1
-score as it equally weighs precision and recall. It is
also possible to give a higher weight to either precision or recall by using a fraction α (with α
between 0 and 1) to indicate the weight for the precision (Do et al., 2002). α=0 indicates that
29

precision is not taken into account at all, while α=1 indicates that the score is entirely based
on the precision. And a value α of 0.5 returns the F
1
-score:
￿ ￿￿￿￿￿￿￿￿￿ ￿
￿￿￿￿￿￿￿￿￿ ￿ ￿￿￿￿￿￿
￿￿ ￿￿￿ ￿ ￿￿￿￿￿￿￿￿￿ ￿￿ ￿ ￿￿￿￿￿￿

As removing inadequate matches is assumed to require less effort than adding missing
matches, an F
2
-score is defined, which gives a higher weight to the recall (with α = 0.25).
Hence, it is considered more important to obtain as many adequate matches as possible. A
value for α of 0.25 results in the following F
2
-score:
￿ ￿￿￿￿￿￿
￿
￿￿￿￿
￿
￿
￿￿￿￿￿￿￿￿￿ ￿ ￿￿￿￿￿￿
￿￿￿￿ ￿ ￿￿￿￿￿￿￿￿￿ ￿￿￿￿￿ ￿ ￿￿￿￿￿￿

Specifically for (schema) matching, an overall score is suggested by Do et al. (2002) that
indicates the match quality including the manual effort needed to adapt the system mapping
into the baseline mapping (i.e. adding false negatives (A) and removing false positives (C) or
post-match effort). Although the measure captures the post-match effort required to transform
the system mapping into the baseline mapping, it cannot clearly separate the required effort
with respect to the obtained results. Therefore, we define a new measure, which we call
revision.

￿￿￿￿￿￿￿￿
￿
￿
￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿
￿￿￿￿￿￿￿￿￿￿￿￿￿￿
￿
￿￿￿
￿￿￿ ￿￿￿￿
￿
Revision
p
represents the fraction of the found matches that has to be removed
afterwards (false positives).
￿￿￿￿￿￿￿￿
￿
￿
￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿
￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿
￿
￿￿￿
￿￿￿ ￿￿￿￿
￿
Revision
n
represents the fraction of the found non-matches (false negatives) that has to
be turned into a match afterwards.
￿￿￿￿￿￿￿￿
￿￿￿￿￿
￿
￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿
￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿
￿
￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿
￿￿￿ ￿￿￿￿ ￿￿￿￿ ￿￿￿￿
￿
￿￿￿ ￿￿￿￿ ￿￿￿￿ ￿￿￿￿ ￿ ￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿ ￿ ￿￿￿￿￿ ￿￿￿ ￿￿￿￿￿￿ ￿￿￿
Revision
total
represents the fraction of the system results (both matches and non-matches)
that has to be adapted in order to obtain the baseline result. Note that in order to determine
all possible matches, we make use of the powerset of the elements (more specifically the
activities) of both models. Powerset ￿(G) represents the set with all possible subsets of graph
G. All possible matches are calculated by taking the Cartesian product of both powersets
(minus the empty sets). Because the elements in the sets of the inadequate matches (false
positives) might overlap with elements of the sets of the unfound matches (false negatives), the
total number of required adaptations is lower than the sum of the inadequate and unfound
30

matches (i.e. lower than |A|+|C|). As an element can only be part of a single match (either
simple or complex), a match including that element can only be part of a single inadequate
and a single unfound match. Revision
total
results in a value between 0 and 1, where 0 reflects
that no additional effort has to be taken (i.e. |A|+|C|= 0), and a value of 1 when the baseline
mapping does not overlap at all with the found system mapping and all baseline matches have
to be added (i.e. |B| = |D| = 0). The same holds for revision
p
and revision
n
, when equal to 0
nothing has to be adapted, and equal to 1, everything has to be adapted. Both pre-match as
well as post-match effort should be determined (Do et al., 2002). Pre-match effort may include
training of machine learning based matchers, configuration of parameters and specification of
auxiliary information (Do et al., 2002). Although revision indicates the relationship between
the number of required changes and the obtained results, it does not capture the human effort
that is required. It is important to evaluate the pre- and post-match effort as it might cancel
out all benefits from automating mapping (Do et al., 2002). For the moment, the pre-match
effort performed by the customer is reduced to the adaptation of models with respect to the
completeness of start- and end-indications of subprocesses. Obviously, several parameters as
mentioned below in the settings section have to be configured and preset, but this will not be
done by the user and will thus not be taken into account for the pre-match effort. Also,
specification of auxiliary information such as synonyms, homonyms and stopwords is not
performed by the customer. The revision measurements revision
p
and revision
total
are used to
provide an indication of the required post-match effort. Again note that this only indicates the
number of matches that has to be adapted versus the number of matches reviewed (which is
equal to all possible matches). It remains difficult to indicate this human effort as it depends
on the cognitive ability and experience of the user.
In our evaluation the following measurements are determined: recall, precision, F
2
-score,
and revision. Additionally, the run-time of the algorithms will be considered.
5.2 Evaluation results
In this section, the evaluation results are described per component.
5.2.1 Searchers
As searchers may be combined, the following searcher sets will be tested (see Table 5.1):
Searcher-set
Similar label match wrapup
searcher
Similar description match
searcher
1
￿￿￿￿

￿￿￿￿

2
￿￿￿￿

￿￿￿￿

3
￿
￿￿
￿

￿
￿￿
￿

4
￿￿￿￿

￿￿￿￿

Table 5.1 Searcher-sets
For the tests additional components are used in order to have a complete matcher (note
that the best selector is used). These are shown in Table 5.2.


Additional component Metric

Booster

-
Selector


Look ahead match similarity selector (0.2)
Evaluator

-
Table 5.2
Additional matcher components for searcher
A small test on one of the model sets was perfor
value for the variables.
These are shown in
Component Metric
Searcher


Similar label match wrapup
searcher

Similar description match
searcher
Table 5.3
Parameter settings searchers
Figure 5.2 Searchers performance
Figure 5.2 shows
that although the label matcher achieves very good results singularly
can be seen from the F
2
-
score which serves as the main performance criteria
the highest precision), the combinati
highest F
2
-score).

32

5.2.2 Boosters
As boosters may be combined, the following booster sets will be tested (see Table 5.4):
Booster-set
Cardinality booster