Software Process

farrightΛογισμικό & κατασκευή λογ/κού

15 Αυγ 2012 (πριν από 4 χρόνια και 10 μήνες)

464 εμφανίσεις

Process and Open Source Software

Walt Scacchi

Institute for Software Research

UC Irvine



http://www.ics.uci.edu/~wscacchi/Presentations/Process/Informatix
-
Process
-
Lecture.ppt


Overview


Core concerns


Software process and process modeling


Backstory/case study


History of software process


Current challenges in software process


Open source software processes


Best practices

Core concerns



Process(es)



What to do
and

how to do it


Tasks, inputs, outputs, performers, tools


Performers enact tasks using tools to transform inputs into
outputs


Tasks are partially ordered


Tasks, inputs, performers, tools may have

pre
-
conditions


Tasks, outputs, performers may have
post
-
conditions


Organize and coordinate who does what, (with what
resources), when, where, how, and why


Formal modeling of processes enables systematic querying and
reasoning about complex processes

Core concerns



Process modeling


Understanding, analyzing, and (re)designing software
development processes


Models may be either


Prescriptive

what
should be

done and how to do it


Proscriptive

what
might be

done and how


Descriptive

what
was

done and how (historic record)



Models may be either


Narrative


Semi
-
structured
(form
-
based; hypermedia
-
based)



Graphic
(attributed flow graphs)



Formal
(language
-
based process programs or specifications)



Families of models inter
-
related through

meta
-
model


Meta
-
model provides model ontology and process epistemology

Backstory/case study


Major TelCo wants to develop broadband multi
-
media
telecommunications system


Anticipates $1B software development, up to 1500
software developers working 2
-
3 years


Seeks industrial/IT partners to provide supporting
infrastructure to reduce risk


IT‏partner‏wants‏to‏showcase‏new‏“process‏support‏
technology”‏products‏as‏sales‏lead


IT partner brings in academic research team to analyze
and‏advise‏TelCo‏on‏“process‏issues”

Backstory


Academic team, IT partner, and major TelCo jointly
elicit, capture, codify (
model
) and inter
-
relate TO
-
BE
system development process.


Academic‏team‏employs‏IT‏partner’s‏products‏to‏
present‏results‏of‏their‏“process‏analysis”


IT partner considers its process
-
centered product major
success to help large sales engagements


Academic team view of their effort
--

a major success
for
publication
, but...

P.K. Garg, P. Mi, T. Pham, W. Scacchi, and G. Thunquest,
The SMART Approach to Software Process Engineering
,

Proc.
16th. Intern. Conf. Software Engineering
, IEEE Computer Society, Sorrento, Italy, 341
-
350, May 1994.

A complex organizational process (and model):


a
decomposition
-
precedence

relationship view
(
19
levels of decomposition,
400
+ tasks)


W. Scacchi,
Experience with Software Process Simulation and Modeling
,
J. Systems and Software
,
46(2/3):183
-
192,1999.

Backstory


Academic team suggests overall process
will not

succeed
--

too complex, too much delegation,
problematic hand
-
offs‏(“throwing‏it‏over‏the‏wall”)



TelCo and IT partner
dismiss

academic team


Less than one year later, IT vendor abandons process
technology product


Two years later, business press reports TelCo
experiences major project failure and losses greater
than $
200
M, and no system.


Software process model history


Process flow charts


Software‏life‏cycle‏(“waterfall”‏diagram)


Incremental development


Spiral model


Process programs (formal specification)


Capability maturity models and industrial/military
standards


Not process models
--
only focus on what to do, but not
how to do it.


Software
Development
Process Model, c.
1955.

Software
Development
Process Model, c.
1960.

Software
Development
Process Model, c.
1970
.


Recent software process
challenges


Distributed, decentralized, and/or global software
development


Process improvement


Process design optimization or redesign


Continuous process improvement (learning)


Articulating software evolution processes


Open source software development processes


Process discovery


(semi
-
)automated extraction of process events,
conditions, timestamps, and other meta
-
data from
software development artifacts

J. Noll and W. Scacchi,
Supporting Software Development in Virtual Enterprises
,
Journal of Digital Information

1
(
4
), February
1999
.



Free/Open Source Software
Development Processes and
Practices

What is free/open source software
development?


Free‏(as‏in‏“freedom”)‏vs.‏open‏source


Freedom to access, browse/view, study, modify and
redistribute the source code


Free is always open, but open is not always free


F/OSSD‏is‏not‏“software‏engineering”


Different
: F/OSSD can be faster, better, and cheaper than
SE in some circumstances


F/OSSD involves
more

software development tools, Web
resources, and personal computing resources


OSS Development Models


Free Software (GPL)


Open Source (BSD/MIT, Mozilla, Apache)


Corporate Source (Hewlett
-
Packard)


Consortium/Alliance (OSDL, SugarCRM)


Corporate
-
Sponsored (IBM
-
Eclipse, Sun
-
Netbeans,
Sun
-
OpenOffice, HP
-
Gelato)


Community Source (Sakai, Westwood)

OSSD Project Characteristics


OSS Developers are always users of what they build,
while OSS users (>
1
%) are also OSS developers


Requires‏“
critical mass
”‏of‏contributors‏and‏OSS‏
components connected through socio
-
technical interaction
networks


OSSD projects emerge/evolve via
bricolage


Unanticipated architectural (de)compositions


Multi
-
project component integrations


OSSD teams use
10
-
50
OSSD tools to support their
development work


OSSD Project Characteristics


Operational code early and often
--
actively
improved and continuously adapted


Post
-
facto

software system requirements and
design


OSSD is not Software Engineering


OSSD‏has‏its‏own‏“
-
ilities”‏which‏differ‏from‏those‏for‏
SE


Caution
: the vast majority of OSSD projects fail to
grow or to produce a beta release.

SourceForge.net

Google Summer of Code 2007

OSS Processes for Requirements or
Design


OSS Requirements/Designs


not explicit


not formal


OSS Requirements/Designs are embedded within
“informalisms”


Example OSS informalisms to follow (as screenshot
displays)


OSS Requirements/Design processes are different
from their SE counterparts
.

SE vs. OSS processes for
Requirements


Elicitation


Analysis





Specification and
modeling


Validation





Communicating and
managing


Post
-
hoc

assertion


Reading, sense
-
making, accountability


Continually emerging
webs of discourse


Condensing and
hardening discourse


Global access to
discourse


Evolutionary redevelopment,
reinvention, and redistribution in OSS


One evolutionary dynamic of OSSD is
reinvention


Reinvention enables continuous improvement


OSS evolve through minor mutations


Expressed, recombined, redistributed via incremental releases


OSS systems
co
-
evolve

with their development
community


Success of one depends on the success of the other


Closed legacy systems may be
revitalized

via
opening and redistribution of their source


When enthusiastic user
-
developers want their cultural experience
with such systems to be maintained.

Configuration management and work
coordination in OSSD


Use CM to coordinate and control who gets to update
what part of the system/online artifacts.


Many OSSD projects use
CVS

(single centralized code
repository with update locks) and frequent releases (
daily
releases

on active projects)


Linux Kernel:
Git

(multiple parallel builds and release
repositories)


Collab.Net and Tigris.org:
Subversion

(CVS++)


Apache:‏Single‏major‏release,‏with‏frequent‏“
patch
”‏
releases‏(e.g.,‏“A‏patchy‏server”)


GNU
arch
seeks to develop Free CM unification

Project management and career
development in OSSD


OSSD projects self
-
organize as a
meritocractic role
-
hierarchy

and
virtual project management


Meritocracies embrace incremental innovations over radical
innovations


VPM requires people to act in leadership roles based on skill,
availability, and belief in project community



OSS developers value
freedom of choice and expression
.


want to learn about new stuff (tools, techniques, skills, etc.),


have fun building software,


exercise their technical skill (e.g., developing reusable code),


try out new kinds of systems to develop, and/or


interconnect multiple F/OSSD projects

(images from A.J. Kim,
Community Building on the Web
,
2000
)


A meritocractic role hierarchy for OSSD

---------------------------

---------------------

Socio
-
technical and cultural evolution
processes in OSSD


New

kinds of OSSD processes under study


Joining

and
contributing to a project

in progress


Role
-
task migration
: from project periphery to center


Alliance formation

and
community development


Independent and autonomous project communities
can interlink via social networks that manipulate
objects of interaction


Enables possible exponential growth of interacting and
interdependent community as
socio
-
technical interaction
network

Discovering the what and how of
OSSD processes


Practitioner reflections and anecdotes


Surveys


Ethnographic field work (virtual ethnography)


Mining OSSD project repositories


Multi
-
modal modeling, analysis, and validation of
OSSD processes

Sun
Microsystems

The Board

Release
Manager

Maintainer

Developers/ Contributors

Website

Mailing Lists

Users

Contribute to
community
,
meet time
constraints for
the release

Maintain a
project/
module,
manage a
group of
developers

Ensure that the
netbeans community
is being run in a fair
and open manner

Start new
release phase,
propose
schedule/plan

CVS

Funds, support,
Promote
Java/Open
source

IssueZilla

QA Team

Produce Q
-

builds and
ensure
quality of
the
software

release proposal
,
release
updates
,
branch for current
release
,
release post
mortem
,
review release
candidates
(
2
) &
decide
final release

download
development
builds and test
,
release Q
-
builds

download new
release

report bugs

select feature to
develop
,
bug to fix
,
download netbeans
,
commit code

decide features for the
project and merge
patches/bug fixes
,
create
module web page

make decisions for
the community, on
high level

grant CVS
commit
privilege to
developers

Link to all Use Cases

Links to all Agents

CVS
Manager

Configur
e and
maintain
CVS

grant
access

Site Administrator

Manage
website

deploy
builds

SourceCast

Tools

Link to Tools

Download and
use free
software

Community
Manager

Share
knowledge
and ensure
all
community
issues are
addressed

respond to tech
issues, unanswered
questions

Rich Picture

NetBeans

Formal model of an OSSD process coded in PML

(excerpt)


...


sequence Test {



action Execute automatic test scripts {



requires { Test scripts, release binaries }



provides { Test results }



tool { Automated test suite (xtest, others) }



agent { Sun ONE Studio QA team }



script { /* Executed off
-
site */ } }


action Execute manual test scripts {



requires { Release binaries }



provides { Test results }



tool { NetBeans IDE }



agent { users, developers, Sun ONE Studio QA team, Sun ONE Studio developers }



script { /* Executed off
-
site */ } }


iteration Update Issuezilla {



action Report issues to Issuezilla {



requires { Test results }



provides { Issuezilla entry }



tool { Web browser }



agent { users, developers, Sun ONE Studio QA team, Sun ONE Studio developers }



script {



<br><a href="http://www.netbeans.org/issues/">Navigate to Issuezilla </a>



<br><a href="http://www.netbeans.org/issues/query.cgi">Query Issuezilla </a>



<br><a href="http://www.netbeans.org/issues/enter_bug.cgi">Enter issue </a> } }




PML validation analysis



Su
m
m
ary

of

a
n
aly
si
s

fo
r

n
e
tb
ea
n
s_re
q
_r
e
l
e
ase
.p
m
l
M
ode
l

si
ze

(
s
ourc
e

li
ne
s
)
:
307
A
c
ti
ons
:
36
R
esou
r
ces
:
72
Ac
ti
on
s
ne
it
he
r

r
equ
iri
ng no
r
p
r
ov
i
d
i
ng
r
e
s
ourc
e
s
:
1
R
e
sou
r
ces
r
equ
i
r
e
d

bu
t
no
t
p
r
ovid
e
d

(
po
t
en
ti
a
l

i
npu
t
s
):
0
R
e
sou
r
ces

p
r
ov
i
ded bu
t
no
t

r
equ
i
r
e
d

(
po
t
en
ti
a
l
ou
t
pu
t
s
):
0
M
ir
ac
l
e
s:
2
B
l
a
c
k

ho
l
e
s:
6
T
ran
s
fo
rm
a
ti
ons
:
30
Best practices


Processes with explicit process models are easier to
manage, analyze, improve, distribute, and reuse


New/ unfamiliar software tools and techniques are best
candidates for software process support


Process meta
-
modeling enables process life cycle
engineering and formal reasoning about processes


OSSD is a
community building process


not just a technical development process


F/OSS peer review creates a
community of peers


OSSD processes often iterate
daily

versus infrequent
singular (milestone) Software Life Cycle Engineering
events


OSSD: frequent, rapid cycle time (
easier

to improve)
vs
.


SLC: infrequent, slow cycle time (
harder

to improve)



Best practices


Easiest to improve a process that is formally
modeled


Process management and improvement have been
one of the most enduring practices in Software
Engineering for improving productivity and
quality, and to reducing cost and risks.

Research
opportunities

Research opportunities


FOSSD is poised to alter the calculus of
empirical SE


Software process discovery, modeling, and
simulation


Repository mining can support software
visualization, refactoring/redesign studies


Comparison of SE versus FOSSD approaches to
software inspection and peer review


Research opportunities


Based on results from individual motivation,
participation, role migration, and turnover in
FOSSD projects, SE world would benefit from
empirical studies that examine similar patterns
in conventional software development projects


Is FOSSD more fun, interesting, and rewarding than
SE?

Research opportunities


Conventional software cost estimation
techniques‏(e.g.,‏“total‏cost‏of‏operation”)‏
slight/ignore social capital and socio
-
technical
resources


Miscalculation of total resources and capabilities
that affect predicted/actual costs of software
development or FOSSD

Research opportunities


Results from study of cooperation, coordination
and control in FOSSD


Virtual project management

and
role migration

processes can provide a lightweight approach to SE
project management


Unclear whether proprietary software projects
willing to embrace VPM

Research opportunities


Alliance formation and social networking results
suggest SE projects operate at a disadvantage
compared to FOSSD projects


SE projects tend to produce systems whose
growth/evolution is limited


FOSSD projects can produce systems capable of
sustained exponential growth/evolution of both
software and developer
-
user community

Research opportunities


How best to encourage the emergence of a
social movement that combines best practices of
FOSSD and SE


Consider participation or study of open source
software engineering (OSSE) projects at Tigris.org


OSSE seeks to combine SE and FOSSD tools,
techniques, and concepts

Detailed study report available


W. Scacchi, Free/Open Source Software
Development: Recent Research Results and
Methods, in M. Zelkowitz (ed.),
Advances in
Computers
, Vol.
69
,
243
-
295
,
2007
.


http://www.ics.uci.edu/~wscacchi/Papers/New/Draft_Chapter_Scacchi.pdf



Acknowledgements


Project collaborators
:


Mark Ackerman, UMichigan, Ann Arbor


Les Gasser, UIllinois, Urbana
-
Champaign


John Noll, Santa Clara University


Margaret Ellliot, Chris Jensen, UCI
-
ISR


and others ar ISR


Funding support
:


National Science Foundation: #
0083075
, #
0205679
, #
0205724
, #
0350754
and #
0534771
.


Digital Industry Promotion (DIP) Agency, Global R&D Collaboration
Center, Daegu, South Korea


No endorsement implied.