Complex Systems Modeling and Networks
Summary of NECSI Course CX202
11 February 2009
John M. Linebarger, PhD
Sandia National Laboratories
Interactive Systems Simulation & Analysis (06344)
Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company,
for the United States Department of Energy’s National Nuclear Security Administration
under contract DE

AC04

94AL85000.
Outline
•
Overview
•
What are Complex
Systems?
•
Complex Networks
–
Metrics
–
Types of Networks
•
Modeling and
Simulation of Complex
Systems
–
How to Model
–
Types of Models
•
Further Reading
•
Class Project
•
Overview of Design for
Tractable Analysis
(DTA) framework
Overview
•
I attended a
Complex Systems Modeling and Networks
(CX202) course from 23

27 June 2008
–
I used all 32 of my FY 2008 Strategic Education Initiative
(SEI) hours on the course; the balance was covered by the
Systems Analysis LDRD.
•
The course was held at the
New England Complex Systems
Institute
(NECSI) on the campus of the Sloan School at MIT,
in Cambridge, MA
•
The course prerequisite is CX201,
Complex Physical,
Biological and Social Systems
–
I fulfilled the prerequisite by reading
Making Things Work
by Yaneer Bar

Yam, and
Linked
by Albert

L
á
szl
ó
Barab
á
si, before the course began
•
The follow

on course is CX203,
Methods for the Study of
Complex Systems
, which is mathematically

oriented
Overview (
cont’d
)
•
Over 30 attendees from all over the world, predominately PhD
students and academics
–
Only three or four people from “industry”
–
My project group consisted of a Norwegian PhD student, a
South Korean health care researcher (working in the Boston
area), an Iranian PhD student (studying in Maryland), and a
Bulgarian PhD student (studying in France)
•
The course was divided into two parts
–
The first part was complex network analysis, taught by Dan
Braha of NECSI and UMass Dartmouth
–
The second part was modeling and simulation of complex
systems, taught by Hiroki Sayama of NECSI and SUNY
Binghamton
•
Project groups met each evening, often until quite late, to work on
their course project
•
Projects (and the associated papers) were presented to the class
on Friday, followed by the final exam for the course
NECSI Course Participants
What are Complex Systems?
•
Complex systems are characterized by emergent, self

organizing, collective behavior
–
Such emergent collective behavior often arises due to the
combination of simple individual behavior patterns
•
The components of a complex system are interdependent
•
Complex systems exhibit multiscale variety
–
The structure of the system differs depending on the level at
which it is viewed
–
A common pattern is a mixture of competition and cooperation
at different levels of the system
•
Complex systems arise due to evolutionary, not
deterministic, processes
–
Engineered systems should be implemented this way too
•
Examples of complex systems include the health care
system, the education system, and recent military conflict
From
Making Things Work
by Yaneer Bar

Yam
Another Characterization of Complex Systems
•
Complex systems contain many constituents
interacting nonlinearly
•
The constituents of a complex system are
interdependent
•
A complex system possesses a structure
spanning several scales
•
A complex system is capable of emerging
behavior
•
Complexity involves an interplay between chaos
and non

chaos
–
“Complex systems dance on the edge of chaos”
•
Complexity involves an interplay between
cooperation and competition
From “Chaos, Complexity, and Entropy” by Michel Baranger
Wicked Problems
•
Every wicked problem is essentially unique
•
There is no definitive formulation of a wicked problem
•
Wicked problems are never solved
•
A wicked problem is not understood until after the
formulation of a solution
•
Solutions to wicked problems change the problem itself
•
Solutions to wicked problems are not true or false, but
better or worse
•
Stakeholders have radically different world views and
different frames for understanding the problem
•
Conclusion
–
Wicked problems are a proper subset of complex
systems
–
All wicked problems are complex systems, but not all
complex systems are wicked problems
Dancing on the Edge of Chaos
•
Two fundamental assumptions of most
systems analysis are that the system is
continuous, and that if it is decomposed
far enough it becomes simple
–
It can then be analyzed without loss
of fidelity
•
These assumptions may be driven by
the predominant analytical tool of 20
th
century physics, which was calculus
–
For both derivatives and integrals, if
you zoom in far enough a curve
becomes a straight line
–
Thus a calculus problem is
transformed into a geometry
problem, which has been well
understood since Euclid
•
However, chaotic systems are often
discontinuous, and the decomposition
of chaotic systems generally yields
more complexity
–
e.g.,
fractals, Sierpi
ń
ski triangles
•
“The arrogance of analysis”
•
Yet chaotic systems often are simple,
not in terms of their decomposition, but
in terms of the rules that govern their
behavior
–
e.g.,
“
boids
,” where simple rules
lead to complex flocking behaviors
Complex Networks
•
The interdependencies of complex systems are often modeled by
networks
•
Networks can be analyzed using several important metrics from
graph theory
–
Directed (nodes and arcs) vs. undirected graphs (nodes and
edges)
–
Number of nodes (N): The number of nodes in a network
–
Degree (k): The number of edges or arcs incident on a node
•
Average degree is denoted as <k>
•
Can distinguish between in degree and out degree
–
Node density is actual connections between nodes divided by
potential connections that could exist between nodes
•
Note that real

world networks are quite sparse, and have an average
density of 10

4
–
Characteristic path length (L): The average length of the shortest
path in the network between two nodes
–
Clustering coefficient (C): Number of actual links between
neighbors of a node divided by the number of possible links
between those neighbors
•
This can be thought of as local density, the probability that two
neighbors know each other
•
The clustering coefficient is a measure of modularity
Random Networks
•
For years (since the late 1950’s)
the dominant model of
networks was the random
network, otherwise known as
the Erd
ő
s

R
é
nyi model, even
though it is rarely seen in
nature
•
Random graphs have a fixed
number of nodes which are
connected randomly by
undirected edges. The number
of edges depend on a specified
probability.
•
The seminal paper is “On
Random Graphs,” by Erd
ő
s and
R
é
nyi (1959)
Small

World Networks
•
Adding a small number of
links to a regular ring network,
in which only the nearest
neighbors are connected, can
have dramatic effects
•
The characteristic path length
becomes quite small with only
a small number of such cross

neighborhood links
•
Small

world networks are
pervasive in nature: Friend
networks,
power grids,
predator

prey networks, and
even neural networks of the
brain.
•
The seminal paper is
“Collective Dynamics of
‘Small

World’ Networks,” by
Watts and Strogatz (1998)
Scale

Free Networks
•
Scale

free networks are so

called because they have no
characteristic scale, or
average number of links per
node
•
Instead, they are characterized
by numerous hubs that hold
together highly connected
regions of the network
•
However, direct connections
between hubs are rare (for
reasons of resilience)
•
Scale

free networks are also
pervasive in nature: Disease
propagation, terrorist
networks, and the World

Wide
Web
•
The seminal paper is
“Emergence of Scaling in
Random Networks,” by
Barab
á
si and Albert (1999)
Scale

Free Networks (
cont’d
)
•
Scale

free networks seem to follow a power law distribution (which is a
variant of a Pareto distribution, or the 80/20 rule)
•
Earlier models (such as Erd
ő
s

R
é
nyi) described networks using a bell
curve, with many nodes with the same number of links and no highly
connected nodes
•
The scale free model describes networks as having many nodes with only
a few links, and a few nodes with many links
•
Power law functions appear as straight lines on a log

log plot
Most nodes have
the same number
of links
No highly connected
nodes
Many nodes with few links
A few nodes with
many links
# of links (k)
# of links (k)
# of nodes with k links
# of nodes with k links
Bell Curve
Power Law Distribution
Scale

Free Networks
(cont’d)
P
k
~ e

k
P
k
~ k

γ
Scale

Free Networks
(cont’d)
•
In a more formal sense, a
scale

free network is
characterized by two
properties
–
Continuous growth (the
other network models did
not consider growth)
–
Preferential attachment
(
e.g.,
“the rich get richer”)
•
The number of nodes with
exactly k links follows a
power law for any value of k
–
P(k) ~ k

γ
–
Often 2 <
γ
< 3
•
However, many real

world
networks actually exhibit
“stretched exponential”
behavior, becoming
exponential past a threshold
–
Cost of adding new links
–
Aging
From Faloutsos, M., P., & C., “On Power

Law Relationships
of the Internet Topology” (1999)
Topology is Destiny!
•
Characterizing a network quantitatively gives insight into the
dynamics (performance) and vulnerabilities of the network
•
Random graphs have a short path length (good), but also a small
clustering coefficient (not good)
•
Regular ring networks have a large clustering coefficient (good) but
long path length (not good)
•
Small

world networks are a balance between those two extremes,
but do not have the robustness of scale

free networks
•
Scale

free networks (which are similar in some ways to small

world
networks) are robust against random attacks (good) but not against
targeted attacks (not good)
–
The hubs are vulnerable when directly targeted
–
The network can become disconnected fairly rapidly in this kind
of attack
•
The moral of the story: When it comes to networks (and to
everything else in life), you can’t have it all …
How Complex Network Research is Done
•
A network under study is represented and analyzed
mathematically
•
A randomly generated “ensemble” of networks is
created with which to compare to the network
under study
–
Ensembles of
several
types of networks are
generated, such as random graphs, small world,
regular ring, and scale

free
•
The network ensembles are analyzed
mathematically and compared to the network under
study
•
Both the network under study and the generated
ensembles are subjected to the same process (
e.g.,
attack), generally via simulation, and the results
are compared
Tools for Complex Network Analysis
•
Pajek
, an open source tool that generates and
visualizes networks
–
A useful book is
Exploratory Social Network
Analysis with Pajek
, by de Nooy
et al
.
•
UCINET
, a commercial tool from Analytic
Technologies that quantitatively analyzes
networks
•
Python
with the
NetworkX
package
•
MATLAB
, a commercial tool from MathWorks,
which can generate, visualize, and quantitatively
analyze networks after “a simple matter of
programming”
–
Free network analysis packages
are available for
MATLAB
Modeling and Simulation of Emergent Behavior
•
Fundamentals of Modeling
•
Modeling and Simulation of Dynamical Systems (complex
interdependent systems that can exhibit emergent behavior)
–
Non

Spatial Modeling
–
Cellular Automata
–
Agent

Based Models
–
Network Models
•
Other Simulation Paradigms for Dynamical Systems (not
covered in the course)
–
Systems Dynamics
•
Numerous commercial packages exist, such as
Vensim
•
Can be described in terms of difference or differential
equations, so can be addressed by non

spatial modeling
–
Spatial models with partial differential equations
–
“Soft computing” (
e.g.,
evolutionary computation, artificial
neural networks, or support vector machines)
Fundamentals of Modeling
•
Scientific “laws” are really hypotheses, models of
observed behavior
•
There is no way to prove anything about nature,
although it is easy to disprove something
•
A model is a simplified representation of a
system
•
Rule

based modeling allows the system’s
possible or future states to be extrapolated,
unlike descriptive modeling, which just specifies
the system’s actual or past states
•
A good model is simple, correct, and robust
Modeling Procedure
•
In general
–
Observe the system of interest
–
Reflect on the possible rules that might cause the
system’s characteristics
–
Derive predictions from those rules and compare them
with reality
–
Repeat to modify the rules until satisfied
•
For complex systems
–
Define the key questions and choose the right scale of
modeling
–
Identify the structure (the parts and their connections) of
the system
–
Define the possible states for each part (state space)
–
Describe how the state of each part changes over time
through interactions with other parts
•
Models must be evaluated and validated, in terms of
simplicity, correctness, and robustness
Tools Used for Modeling and Simulation
•
Python
•
NumPy and SciPy
(MATLAB

like numerical
packages for Python)
•
matplotlib
(MATLAB

like visualization library for
Python)
•
NetworkX
(network graphing package)
•
Agent

based modeling toolkits, such as
NetLogo
,
StarLogo
, or
Swarm
(see the
Swarm Wiki
for
other tools)
•
And, of course,
MATLAB
…
Non

Spatial Modeling
•
The simplest mathematical model
–
Called “non

spatial” because the spatial location of the
elements of the system is unimportant
•
A dynamical system is a system whose state is uniquely
specified by a finite set of variables and whose behavior is
uniquely determined by predetermined rules
–
Examples are simple population growth, simple pendulum
swing, motion of celestial bodies
•
Two schools of modeling dynamical systems
–
Discrete

time model (using difference/recurrence
equations or iterative maps)
–
Continuous

time model (using differential equations)
•
Non

autonomous, higher

order equations can always be
converted into autonomous, first order equations, by
introducing additional variables
•
Non

linear systems are generally simulated because they
cannot be solved analytically
•
Exercise: Logistic growth (logisticgrowthmodel.py)
Cellular Automata
•
Adds spatio

temporal dynamics, especially for
locally connected networks, to the modeling
process
•
Cells in a regular grid interact with their nearest
neighbors according to simple interaction rules
–
Transition from one state to another depends on
simple rules based on interactions with neighbors
–
The resulting system is a linear system
•
Example: Simple diffusion, panic in a gym
•
Exercise: Decide whether or not to panic based
on the state of a simple majority of your
neighbors (majority.py)
Agent

Based Models
•
The only way to study the behavior of agent

based models is to simulate, since no global
equation exists, only equations at the agent level
•
Agent characteristics (what makes an agent an
agent)
–
Local state
–
Spatial localization
–
Local interaction with other agents based on
predefined rules
–
No central supervisor
–
May learn autonomously
–
May produce non

trivial collective behavior
Agent

Based Models (
cont’d
)
•
To develop an agent

based model
–
Identify the agents and get a theory of agent behavior
–
Identify the agent relationships and get a theory of agent
interaction
–
Get the requisite agent

related data
–
Validate the agent behavior models in addition to the model as
a whole (thus you must justify your model relative to the real

world, including emergent behavior)
–
Run the model and analyze the output
•
Exercise: Garbage collection by ants (ants.py)
–
Eventually, only one pile of garbage remains
–
No simulation of the environment is needed, just tracking of
the environment; however, if pheremones were emitted and
decayed with time, the environment would need to be
simulated as well, based on a model of pheremone decay
•
An excellent two

part tutorial was presented by Macal and North at
the Winter Simulation Conference in 2005 and 2006, respectively
Network Models
•
Simulation of the types of complex networks we
learned about in the first part of the course
•
Uses the
NetworkX
package from LANL
–
Has methods to create random networks, small

world networks, and scale

free networks
•
Exercise: A disease epidemic on a random
network (networkepidemics.py)
–
Two parameters: Infection probability and recovery
probability
–
Red means infected; yellow means susceptible
–
Note that the disease is never completely
eradicated from a social network; this is called the
fixation of a disease
–
Can experiment with different network topologies
Further Reading
•
Complexity and Chaos
–
Making Things Work
, by Yaneer Bar

Yam (2005)
–
Ubiquity: The Science of History … or Why the World is Simpler
than We Think
(a.k.a. “
Ubiquity: Why Catastrophes Happen
”), by
Mark Buchanan (2001, 2002)
–
Chaos: Making a New Science
, by James Gleick (1988)
–
The Dynamics of Complex Systems
, by Yaneer Bar

Yam (1997)
•
Complex networks
–
Linked
, by Albert

L
á
szl
ó
Barab
á
si (2003)
–
Small Worlds: The Dynamics of Networks between Order and
Randomness
, by D. J. Watts (2003)
–
Exploratory Social Network Analysis with Pajek
, by de Nooy,
et al
.
(2005)
•
Modeling and simulation of dynamical systems
–
Cellular Automata
, by Andrew Ilachinski (2001)
–
Python Scripting for Computational Science
, 3
rd
ed., by Hans Petter
Langtangen (2006)
–
Simulation Modeling and Analysis
, 4
th
ed., by Averill Law (2007)
Class Project
Comments 0
Log in to post a comment