Complex Systems Modeling and Networks

rucksackbulgeAI and Robotics

Dec 1, 2013 (3 years and 4 months ago)

56 views

Complex Systems Modeling and Networks

Summary of NECSI Course CX202


11 February 2009


John M. Linebarger, PhD

Sandia National Laboratories

Interactive Systems Simulation & Analysis (06344)

Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company,

for the United States Department of Energy’s National Nuclear Security Administration


under contract DE
-
AC04
-
94AL85000.

Outline


Overview


What are Complex
Systems?


Complex Networks


Metrics


Types of Networks


Modeling and
Simulation of Complex
Systems


How to Model


Types of Models


Further Reading


Class Project


Overview of Design for
Tractable Analysis
(DTA) framework

Overview


I attended a
Complex Systems Modeling and Networks
(CX202) course from 23
-
27 June 2008


I used all 32 of my FY 2008 Strategic Education Initiative
(SEI) hours on the course; the balance was covered by the
Systems Analysis LDRD.


The course was held at the
New England Complex Systems
Institute
(NECSI) on the campus of the Sloan School at MIT,
in Cambridge, MA


The course prerequisite is CX201,
Complex Physical,
Biological and Social Systems


I fulfilled the prerequisite by reading
Making Things Work

by Yaneer Bar
-
Yam, and
Linked

by Albert
-
L
á
szl
ó
Barab
á
si, before the course began


The follow
-
on course is CX203,
Methods for the Study of
Complex Systems
, which is mathematically
-
oriented

Overview (
cont’d
)


Over 30 attendees from all over the world, predominately PhD
students and academics


Only three or four people from “industry”


My project group consisted of a Norwegian PhD student, a
South Korean health care researcher (working in the Boston
area), an Iranian PhD student (studying in Maryland), and a
Bulgarian PhD student (studying in France)


The course was divided into two parts


The first part was complex network analysis, taught by Dan
Braha of NECSI and UMass Dartmouth


The second part was modeling and simulation of complex
systems, taught by Hiroki Sayama of NECSI and SUNY
Binghamton


Project groups met each evening, often until quite late, to work on
their course project


Projects (and the associated papers) were presented to the class
on Friday, followed by the final exam for the course

NECSI Course Participants

What are Complex Systems?


Complex systems are characterized by emergent, self
-
organizing, collective behavior


Such emergent collective behavior often arises due to the
combination of simple individual behavior patterns


The components of a complex system are interdependent


Complex systems exhibit multiscale variety


The structure of the system differs depending on the level at
which it is viewed


A common pattern is a mixture of competition and cooperation
at different levels of the system


Complex systems arise due to evolutionary, not
deterministic, processes


Engineered systems should be implemented this way too


Examples of complex systems include the health care
system, the education system, and recent military conflict

From
Making Things Work

by Yaneer Bar
-
Yam

Another Characterization of Complex Systems


Complex systems contain many constituents
interacting nonlinearly


The constituents of a complex system are
interdependent


A complex system possesses a structure
spanning several scales


A complex system is capable of emerging
behavior


Complexity involves an interplay between chaos
and non
-
chaos


“Complex systems dance on the edge of chaos”


Complexity involves an interplay between
cooperation and competition

From “Chaos, Complexity, and Entropy” by Michel Baranger

Wicked Problems


Every wicked problem is essentially unique


There is no definitive formulation of a wicked problem


Wicked problems are never solved


A wicked problem is not understood until after the
formulation of a solution


Solutions to wicked problems change the problem itself


Solutions to wicked problems are not true or false, but
better or worse


Stakeholders have radically different world views and
different frames for understanding the problem


Conclusion


Wicked problems are a proper subset of complex
systems


All wicked problems are complex systems, but not all
complex systems are wicked problems

Dancing on the Edge of Chaos


Two fundamental assumptions of most
systems analysis are that the system is
continuous, and that if it is decomposed
far enough it becomes simple


It can then be analyzed without loss
of fidelity


These assumptions may be driven by
the predominant analytical tool of 20
th

century physics, which was calculus


For both derivatives and integrals, if
you zoom in far enough a curve
becomes a straight line


Thus a calculus problem is
transformed into a geometry
problem, which has been well
understood since Euclid


However, chaotic systems are often
discontinuous, and the decomposition
of chaotic systems generally yields
more complexity


e.g.,

fractals, Sierpi
ń
ski triangles


“The arrogance of analysis”


Yet chaotic systems often are simple,
not in terms of their decomposition, but
in terms of the rules that govern their
behavior


e.g.,


boids
,” where simple rules
lead to complex flocking behaviors

Complex Networks


The interdependencies of complex systems are often modeled by
networks


Networks can be analyzed using several important metrics from
graph theory


Directed (nodes and arcs) vs. undirected graphs (nodes and
edges)


Number of nodes (N): The number of nodes in a network


Degree (k): The number of edges or arcs incident on a node


Average degree is denoted as <k>


Can distinguish between in degree and out degree


Node density is actual connections between nodes divided by
potential connections that could exist between nodes


Note that real
-
world networks are quite sparse, and have an average
density of 10
-
4


Characteristic path length (L): The average length of the shortest
path in the network between two nodes


Clustering coefficient (C): Number of actual links between
neighbors of a node divided by the number of possible links
between those neighbors


This can be thought of as local density, the probability that two
neighbors know each other


The clustering coefficient is a measure of modularity

Random Networks



For years (since the late 1950’s)
the dominant model of
networks was the random
network, otherwise known as
the Erd
ő
s
-
R
é
nyi model, even
though it is rarely seen in
nature


Random graphs have a fixed
number of nodes which are
connected randomly by
undirected edges. The number
of edges depend on a specified
probability.


The seminal paper is “On
Random Graphs,” by Erd
ő
s and
R
é
nyi (1959)

Small
-
World Networks


Adding a small number of
links to a regular ring network,
in which only the nearest
neighbors are connected, can
have dramatic effects


The characteristic path length
becomes quite small with only
a small number of such cross
-
neighborhood links


Small
-
world networks are
pervasive in nature: Friend
networks,

power grids,
predator
-
prey networks, and
even neural networks of the
brain.


The seminal paper is
“Collective Dynamics of
‘Small
-
World’ Networks,” by
Watts and Strogatz (1998)

Scale
-
Free Networks


Scale
-
free networks are so
-
called because they have no
characteristic scale, or
average number of links per
node


Instead, they are characterized
by numerous hubs that hold
together highly connected
regions of the network


However, direct connections
between hubs are rare (for
reasons of resilience)


Scale
-
free networks are also
pervasive in nature: Disease
propagation, terrorist
networks, and the World
-
Wide
Web


The seminal paper is
“Emergence of Scaling in
Random Networks,” by
Barab
á
si and Albert (1999)

Scale
-
Free Networks (
cont’d
)


Scale
-
free networks seem to follow a power law distribution (which is a
variant of a Pareto distribution, or the 80/20 rule)


Earlier models (such as Erd
ő
s
-
R
é
nyi) described networks using a bell
curve, with many nodes with the same number of links and no highly
connected nodes


The scale free model describes networks as having many nodes with only
a few links, and a few nodes with many links


Power law functions appear as straight lines on a log
-
log plot

Most nodes have
the same number
of links

No highly connected
nodes

Many nodes with few links

A few nodes with
many links

# of links (k)

# of links (k)

# of nodes with k links

# of nodes with k links

Bell Curve

Power Law Distribution

Scale
-
Free Networks
(cont’d)

P
k

~ e
-
k

P
k

~ k
-
γ

Scale
-
Free Networks
(cont’d)


In a more formal sense, a
scale
-
free network is
characterized by two
properties


Continuous growth (the
other network models did
not consider growth)


Preferential attachment
(
e.g.,

“the rich get richer”)


The number of nodes with
exactly k links follows a
power law for any value of k


P(k) ~ k
-
γ


Often 2 <
γ

< 3


However, many real
-
world
networks actually exhibit
“stretched exponential”
behavior, becoming
exponential past a threshold


Cost of adding new links


Aging

From Faloutsos, M., P., & C., “On Power
-
Law Relationships
of the Internet Topology” (1999)

Topology is Destiny!


Characterizing a network quantitatively gives insight into the
dynamics (performance) and vulnerabilities of the network


Random graphs have a short path length (good), but also a small
clustering coefficient (not good)


Regular ring networks have a large clustering coefficient (good) but
long path length (not good)


Small
-
world networks are a balance between those two extremes,
but do not have the robustness of scale
-
free networks


Scale
-
free networks (which are similar in some ways to small
-
world
networks) are robust against random attacks (good) but not against
targeted attacks (not good)


The hubs are vulnerable when directly targeted


The network can become disconnected fairly rapidly in this kind
of attack


The moral of the story: When it comes to networks (and to
everything else in life), you can’t have it all …

How Complex Network Research is Done


A network under study is represented and analyzed
mathematically


A randomly generated “ensemble” of networks is
created with which to compare to the network
under study


Ensembles of
several

types of networks are
generated, such as random graphs, small world,
regular ring, and scale
-
free


The network ensembles are analyzed
mathematically and compared to the network under
study


Both the network under study and the generated
ensembles are subjected to the same process (
e.g.,

attack), generally via simulation, and the results
are compared

Tools for Complex Network Analysis


Pajek
, an open source tool that generates and
visualizes networks


A useful book is
Exploratory Social Network
Analysis with Pajek
, by de Nooy
et al
.


UCINET
, a commercial tool from Analytic
Technologies that quantitatively analyzes
networks


Python

with the
NetworkX

package


MATLAB
, a commercial tool from MathWorks,
which can generate, visualize, and quantitatively
analyze networks after “a simple matter of
programming”


Free network analysis packages
are available for
MATLAB

Modeling and Simulation of Emergent Behavior


Fundamentals of Modeling


Modeling and Simulation of Dynamical Systems (complex
interdependent systems that can exhibit emergent behavior)


Non
-
Spatial Modeling


Cellular Automata


Agent
-
Based Models


Network Models


Other Simulation Paradigms for Dynamical Systems (not
covered in the course)


Systems Dynamics


Numerous commercial packages exist, such as
Vensim


Can be described in terms of difference or differential
equations, so can be addressed by non
-
spatial modeling


Spatial models with partial differential equations


“Soft computing” (
e.g.,

evolutionary computation, artificial
neural networks, or support vector machines)

Fundamentals of Modeling


Scientific “laws” are really hypotheses, models of
observed behavior


There is no way to prove anything about nature,
although it is easy to disprove something


A model is a simplified representation of a
system


Rule
-
based modeling allows the system’s
possible or future states to be extrapolated,
unlike descriptive modeling, which just specifies
the system’s actual or past states


A good model is simple, correct, and robust

Modeling Procedure


In general


Observe the system of interest


Reflect on the possible rules that might cause the
system’s characteristics


Derive predictions from those rules and compare them
with reality


Repeat to modify the rules until satisfied


For complex systems


Define the key questions and choose the right scale of
modeling


Identify the structure (the parts and their connections) of
the system


Define the possible states for each part (state space)


Describe how the state of each part changes over time
through interactions with other parts


Models must be evaluated and validated, in terms of
simplicity, correctness, and robustness

Tools Used for Modeling and Simulation


Python


NumPy and SciPy

(MATLAB
-
like numerical
packages for Python)


matplotlib

(MATLAB
-
like visualization library for
Python)


NetworkX

(network graphing package)


Agent
-
based modeling toolkits, such as
NetLogo
,
StarLogo
, or
Swarm

(see the
Swarm Wiki

for
other tools)


And, of course,
MATLAB



Non
-
Spatial Modeling


The simplest mathematical model


Called “non
-
spatial” because the spatial location of the
elements of the system is unimportant


A dynamical system is a system whose state is uniquely
specified by a finite set of variables and whose behavior is
uniquely determined by predetermined rules


Examples are simple population growth, simple pendulum
swing, motion of celestial bodies


Two schools of modeling dynamical systems


Discrete
-
time model (using difference/recurrence
equations or iterative maps)


Continuous
-
time model (using differential equations)


Non
-
autonomous, higher
-
order equations can always be
converted into autonomous, first order equations, by
introducing additional variables


Non
-
linear systems are generally simulated because they
cannot be solved analytically


Exercise: Logistic growth (logisticgrowthmodel.py)

Cellular Automata


Adds spatio
-
temporal dynamics, especially for
locally connected networks, to the modeling
process


Cells in a regular grid interact with their nearest
neighbors according to simple interaction rules


Transition from one state to another depends on
simple rules based on interactions with neighbors


The resulting system is a linear system


Example: Simple diffusion, panic in a gym


Exercise: Decide whether or not to panic based
on the state of a simple majority of your
neighbors (majority.py)

Agent
-
Based Models


The only way to study the behavior of agent
-
based models is to simulate, since no global
equation exists, only equations at the agent level


Agent characteristics (what makes an agent an
agent)


Local state


Spatial localization


Local interaction with other agents based on
predefined rules


No central supervisor


May learn autonomously


May produce non
-
trivial collective behavior

Agent
-
Based Models (
cont’d
)


To develop an agent
-
based model


Identify the agents and get a theory of agent behavior


Identify the agent relationships and get a theory of agent
interaction


Get the requisite agent
-
related data


Validate the agent behavior models in addition to the model as
a whole (thus you must justify your model relative to the real
-
world, including emergent behavior)


Run the model and analyze the output


Exercise: Garbage collection by ants (ants.py)


Eventually, only one pile of garbage remains


No simulation of the environment is needed, just tracking of
the environment; however, if pheremones were emitted and
decayed with time, the environment would need to be
simulated as well, based on a model of pheremone decay


An excellent two
-
part tutorial was presented by Macal and North at
the Winter Simulation Conference in 2005 and 2006, respectively

Network Models


Simulation of the types of complex networks we
learned about in the first part of the course


Uses the
NetworkX

package from LANL


Has methods to create random networks, small
-
world networks, and scale
-
free networks


Exercise: A disease epidemic on a random
network (networkepidemics.py)


Two parameters: Infection probability and recovery
probability


Red means infected; yellow means susceptible


Note that the disease is never completely
eradicated from a social network; this is called the
fixation of a disease


Can experiment with different network topologies

Further Reading


Complexity and Chaos


Making Things Work
, by Yaneer Bar
-
Yam (2005)


Ubiquity: The Science of History … or Why the World is Simpler
than We Think

(a.k.a. “
Ubiquity: Why Catastrophes Happen
”), by
Mark Buchanan (2001, 2002)


Chaos: Making a New Science
, by James Gleick (1988)


The Dynamics of Complex Systems
, by Yaneer Bar
-
Yam (1997)


Complex networks


Linked
, by Albert
-
L
á
szl
ó
Barab
á
si (2003)


Small Worlds: The Dynamics of Networks between Order and
Randomness
, by D. J. Watts (2003)


Exploratory Social Network Analysis with Pajek
, by de Nooy,
et al
.
(2005)


Modeling and simulation of dynamical systems


Cellular Automata
, by Andrew Ilachinski (2001)


Python Scripting for Computational Science
, 3
rd

ed., by Hans Petter
Langtangen (2006)


Simulation Modeling and Analysis
, 4
th

ed., by Averill Law (2007)

Class Project