and John Krogstie:

parsimoniousknotΔίκτυα και Επικοινωνίες

16 Φεβ 2014 (πριν από 3 χρόνια και 8 μήνες)

95 εμφανίσεις

1

Sundar Gopalakrishnan, Guttorm Sindre,

and John Krogstie:

Adapting UML activity diagrams for mobile

work process modelling:

Experimental comparison of two notation alternatives


PoEM 2010 Delft, 9.
-
10. Sept.

2

Agenda


Motivation


Background


Research method


Results


Discussion of threats to validity


Summary and further work

3

Traditional

perspectives

to
modeling


Structural


Behavioral


Functional


Goal and rule
-
oriented


Object
-
oriented


Social communication


Actor/role
-
oriented


What

about

location/place/space
... ?

4



Zachman Framework

21
e.g. DATA
ENTERPRISE ARCHITECTURE - A FRAMEWORK
Builder
SCOPE
(CONTEXTUAL)
MODEL
(CONCEPTUAL)
ENTERPRISE
Designer
SYSTEM
MODEL
(LOGICAL)
TECHNOLOGY
MODEL
(PHYSICAL)
DETAILED
REPRESEN-
TATIONS
(OUT-OF-
CONTEXT)
Sub-
Contractor
FUNCTIONING
ENTERPRISE
DATA
FUNCTION
NETWORK
e.g. Data Definition
Ent = Field
Reln = Address
e.g. Physical Data Model
Ent = Segment/Table/etc.
Reln = Pointer/Key/etc.
e.g. Logical Data Model
Ent = Data Entity
Reln = Data Relationship
e.g. Semantic Model
Ent = Business Entity
Reln = Business Relationship
List of Things Important
to the Business
ENTITY = Class of
Business Thing
List of Processes the
Business Performs
Function = Class of
Business Process
e.g. Application Architecture
I/O = User Views
Proc .= Application Function
e.g. System Design
I/O = Data Elements/Sets
Proc.= Computer Function
e.g. Program
I/O = Control Block
Proc.= Language Stmt
e.g. FUNCTION
e.g. Business Process Model
Proc. = Business Process
I/O = Business Resources
List of Locations in which
the Business Operates
Node = Major Business
Location
e.g. Business Logistics
System
Node = Business Location
Link = Business Linkage
e.g. Distributed System
Node = I/S Function
(Processor, Storage, etc)
Link = Line Characteristics
e.g. Technology Architecture
Node = Hardware/System
Software
Link = Line Specifications
e.g. Network Architecture
Node = Addresses
Link = Protocols
e.g. NETWORK
Architecture
Planner
Owner
Builder
ENTERPRISE
MODEL
(CONCEPTUAL)
Designer
SYSTEM
MODEL
(LOGICAL)
TECHNOLOGY
MODEL
(PHYSICAL)
DETAILED
REPRESEN-
TATIONS
(OUT-OF
CONTEXT)
Sub-
Contractor
FUNCTIONING
MOTIVATION
TIME
PEOPLE
e.g. Rule Specification
End = Sub-condition
Means = Step
e.g. Rule Design
End = Condition
Means = Action
e.g., Business Rule Model
End = Structural Assertion
Means =Action Assertion
End = Business Objective
Means = Business Strategy
List of Business Goals/Strat
Ends/Means=Major Bus. Goal/
Critical Success Factor
List of Events Significant
Time = Major Business Event
e.g. Processing Structure
Cycle = Processing Cycle
Time = System Event
e.g. Control Structure
Cycle = Component Cycle
Time = Execute
e.g. Timing Definition
Cycle = Machine Cycle
Time = Interrupt
e.g. SCHEDULE
e.g. Master Schedule
Time = Business Event
Cycle = Business Cycle
List of Organizations
People = Major Organizations
e.g. Work Flow Model
People = Organization Unit
Work = Work Product
e.g. Human Interface
People = Role
Work = Deliverable
e.g. Presentation Architecture
People = User
Work = Screen Format
e.g. Security Architecture
People = Identity
Work = Job
e.g. ORGANIZATION
Planner
Owner
to the Business
Important to the Business
What
How
Where
Who
When
Why
John A. Zachman, Zachman International (810) 231-0531
SCOPE
(CONTEXTUAL)
Architecture
e.g. STRATEGY
ENTERPRISE
e.g. Business Plan
TM
5

Motivation



Where
’ is
increasingly

relevant


Outsourcing


Supply

Chain

Management/Logistics


Virtual

organization


Mobile
applications

and
information

systems


And it is
possible

to
utilize


where
’ to a
larger

degree

(
also

real
time) to know
where

users
,
equipment

and
goods

should

be,
are

or
where

at a
certain

time


Tracking

(RFID, UWB, GPS, GSM,
WiFi
, Ultrasound…)


Internet
of

Things

(IoT)



This
paper
:


Presents
some

different

notation

alternatives
based

on

UML
activity

diagrams vs.
Modeling

mobile
information

systems


Summarizes

an
analytical

evaluation

from a
previous

paper

(I
-
ESA’10)


Makes an
experimental

comparison

of

the

two

most
promising

ones






6

Possible notations looked at

(I
-
ESA’10)


Standard UML, using annotation boxes to indicate
context or location


Redefining swimlanes to indicate context / location


Using colour for context / location



These three were compared analytically, using a
home care case supported by a mobile IS as an
example

7

Alt 1: Using annotation boxes


Advantages


Smallest deviation from standard
UML AD


Enhanced understandability for those
who already know AD


Disadvantages


Greatly increases # nodes in diagram
(poor expressive economy)


May be confusing if you also need to
use notes for something else in
addition to context / location


8

Alt 2: Swimlanes for where


Advantages


Shifting ”who” to stick figures:
AD more uniform with UCD


Swimlanes intuitively indicate
location


Disadvantages


Many lines from stick figures to
activities,
poor readability
(would be even worse with
bigger example)


New usage of swimlanes may
confuse those already familiar
with AD


9

Alt 3.: using colour


Advantages


No increase in # nodes or
lines, better readability


With two process design
alternatives beside each
other, it is easy to spot
differences in location


Disadvantages


Larger deviation from
standard UML


Must add legend


Possible challenge for colour
blind users

10

Analytical comparison

Analytical comparison (I
-
ESA’10):


Alt 1. (trad. UML with annotations) and Alt 3.
(Colours) came out as the two most promising


Proposed further work: Make experimental
comparison

Notation

Minimal deviation from
standard

Expressiveness

Intuitive / Easy to read

Less Complexity

Simple

Large

Simple

Large

Simple

Large

Simple

Large

Annotated

+

+

+

+


-


-

-


+


-

-

Location
Swimlanes

-


-

+

+


-


--


+


-

-

Colours

-

-

+ +

+ +

+

+

+ +

+ +

11

Experimental design


Compare two diagram alternatives, annotation and colour


Controlled experiment looking at the participants’


Performance using the notation


Opinion about the notation


Within
-
subjects design (Latin squares)


Controls better for selection bias


”Doubles” the N


Measured variables:


Performance:


Understanding
: score on 12 True/False questions about a case after reading textual
description and seeing diagram


Error_detection:
score on identifying 5 deliberately seeded errors in a diagram relative
to textual description


Opinion:


Perceived Ease of Use, Perceived Usefulness
, and
Intention to Use,

measured by
answers to a TAM
-
inspired questionnaire w 14 questions


12

Hypotheses


Since the colour notation was best in the analytical
comparison, this was hypothesized to have
advantages, i.e.:


H1
: understanding scores will be better for the colour notation than
for the annotated notation


H2
: error detection scores will be better for the colour notation than
for the annotated notation


H3
: participants’ opinion about the colour notation will be more
positive than for the annotated notation

13

Experimental tasks

1.
Answering a pre
-
experiment questionnaire
investigating relevant competence

2.
Reading tutorial about first diagram notation
(annotatated or colour depending on group)

3.
Reading text and diagram for case (home care or
traffic control), answering 12 T/F questions

4.
Answering post
-
task questionnaire giving opinion
about the notation

5.
Repeating steps 2
-
4 with the opposite case and
notation

6.
Repeating 2
-
6 with the error detection task


14

Latin squares design

Group

Id

(Understanding+

TAM

factor)

Questionnaire

on

Error Identification
Questionnaire on

Group

A

Annotated

Home

Care

+

Colour

Traffic

Control

Annotated

traffic

Control

+

Colour

Home

Care

Group

B

Colour

Traffic

Control

+

Annotated

Home

Care

Colour

Home

Care

+

Annotated

traffic

Control

Group

C

Annotated

traffic

Control

+

Colour

Home

Care

Annotated

Home

Care

+

Colour

Traffic

Control

Group

D

Colour

Home

Care

+

Annotated

traffic

Control

Colour

Traffic

Control

+

Annotated

Home

Care

15

Results


46 students participated, randomly assigned into the
four Latin squares groups


Clear advantage for colour notation in performance


3 students performed very poorly in error detection (not doing a
serious job); might be considered outliers?


Slight, but not significant advantage for colour when it
comes to opinion



Compared

variable

(N=
46
)

Coloured

diagram

Annotated

diagram

Diffe
-
rence

Effect

Size

Sign
.
?

Y/N

(p
-
value)

Mean

SD

Mean

SD

Understanding

0
.
960

0
.
057

0
.
926

0
.
078

0
.
0344

0
.
51

Y

(
0
.
01
)

Error

detection

4
.
50

1
.
34

3
.
93

1
.
17

0
.
57

0
.
45

Y

(
0
.
001
)

Errors (w/o
outliers, N=43)

4
.
77

0
.
53

4
.
16

1
.
04

0
.
60

0
.
77

Y

(
0
.
001
)

16

Conclusion on hypotheses


H1: coloured notation would be better than
annotated for understanding (answering T/F
questions): CONFIRMED


H2: coloured notation would be better than annotated
for detecting errors: CONFIRMED


H3: participants’ opinion about the coloured notation
would be more positive: REJECTED


17

Threats to validity


Conclusion validity: significant results but small to
moderate effects, should have had larger N to make
a strong claim about results


Construct validity: many other ways of understanding
a model than answering T/F questions, and many
other work tasks than identifying errors. But at least,
this is a relevant task, and the ability to answer
questions correctly should indicate to some extent
whether a model has been understood


18

Threats to validity, cont.


Internal validity: Latin squares design and pre
-
exp.
questionnaire should control very well for any
selection bias. Notations were presented in equal
detail and style in tutorials, and no preferred or
hypothesized outcome was signalled to the students


External validity: The biggest challenge:


Students are not practitioners and motivation may be limited in an
experiment which has no impact on their job or a delivered product.
But the comparative nature of the experiment should mean that
performance with both notations are equally hurt by low
competence or motivation.


Small experimental tasks are not representative of the more
complex tasks in ”real” mobile IS development.

19

Further work


Also experiment with different pattern fills instead of
colour (e.g. better for colour blind users)


More experiments, possibly including practitioners
and/or including collaboration among several persons
instead of just individuals answering questions


Larger industrial case studies, to try out alternative
notations with larger and more realistic work tasks