The Historical Evolution of Modern Systems Biology

tripastroturfΤεχνίτη Νοημοσύνη και Ρομποτική

7 Νοε 2013 (πριν από 3 χρόνια και 9 μήνες)

73 εμφανίσεις

The Historical Evolution of Modern Systems Biology


1 Introduction


Biological phenomena are among the most complex phenomena known to exist. Even
the smallest cell or the smallest related set of biochemical reactions consists of many
diverse elements tha
t engage in numerous, complicated, and often incompletely
understood interactions with one another. Because of this complexity, it is vital that
biology adopt an approach that integrates these diverse elements and complex
interactions into a unified frame
work. However, attempts at such integration have
achieved limited success in the past, due to the complexity of the natural world and
computational limitations.


Fortunately, some of the problems faced by modern systems biologists are not without
preceden
t. The idea of viewing a collection of elements and their interactions with one
another as an integrated whole, or a system, is not a new idea. From Newton’s dynamical
systems to modern systems theory, where elements are expressed by nonlinear
differenti
al equations and stochastic methods, systems ideas have been applied to many
phenomena and have evolved with each application [Figure 1]. An examination of the
past applications of systems ideas, the specific issues that spurred each development, and
the
conditions that led to either the success or failure of the application will provide a
roadmap of sorts to modern systems biologists and will hopefully provide some insight
on the unique problems faced by systems biology.




Figure 1:
Knowledge Flow Durin
g the Evolution of Systems Biology


2 The Foundations of Systems Ideas


The fundamental components of modern systems theory are four millennia old. Chinese
medicine applied systems ideas to physiology as early as 2500 B.C. [0.5], and the idea of
treating
a body as a unified whole was also popular with Western physiologists of the
ancient world. The earliest recorded attempts at formulating a theory of systems in the
Western world were those of Aristotle and other Greek philosophers, which occurred
around
300 B.C. Aristotle proposed that nature was made up of primary, indivisible
constituents and that these constituents possessed both intrinsic properties and extrinsic
properties, or interactions [1]. Aristotle's idea of teleological causation, which post
ulated
that organisms have natural goals and that evolution can only be understood in terms of
these goals, is still one of the fundamental assumptions of biology [2].


Mathematical rigor was not introduced into systems thinking until Newton's work on
dyna
mical systems, which placed all system behavior within a set of cause
-
and
-
effect
rules [3]. Originally developed for astronomy, the idea of dynamical systems migrated to
engineering, thermodynamics, ecology, and genetics, and evolved with each application

to form some of the fundamental components of modern systems theory. Computational
difficulties stemming from both system size and system complexity first appeared in
these applications, and this prompted the development of mathematical techniques that
a
re still in use in the present day.




Figure 2:

A Timeline of Key Events During the Classical Era


2.1 Newtonian Physics and Engineering


Newtonian physics and engineering provided the most fertile ground for systems
thinking. The advantageous condition
s provided by early Newtonian physics and
engineering stemmed from the popular scientific and philosophical paradigm of the
Renaissance and Industrial Revolution popularized by Descartes and other philosophers,
which held that the universe operated in a me
chanistic fashion and all behavior was a
result of cause
-
and
-
effect interactions [4]. Given this viewpoint, it is not surprising that
Newtonian physics and engineering would be the first fields to provide a general, if
rudimentary, mathematical study of s
ystems.


Physics and engineering of the time possessed several advantages that contributed to the
successful application of systems ideas. Due to the lack of availability of precise
instruments, the only systems that could be effectively studied were

those composed of
obviously discrete components. In addition, most systems studied by these fields were
composed of relatively few elements. In the case of those systems that were composed of
a larger number of elements, the resolution of the available
instruments was such that
many interactions of small magnitude could be ignored. The end results of these three
advantages were simple models of the systems in question, which enabled the systems to
be studied as a whole with ease and accuracy relative to

observation.


Engineering possessed additional advantages that some natural systems did not. While
philosophers grappled with the nature of God and the purpose of the universe, engineers
designed their systems for a known, predefined purpose of their own

choosing. This
allowed them to fully specify every element, in some cases arbitrarily define elements,
artificially constrain the number of interactions or remove unwanted interactions to
ensure a level of simplicity, and also provided a framework for un
derstanding the
evolution of the state of the system. These advantages account for the continued success
of such things as industrial design techniques [5] and control theory [6] today.


2.2 Thermodynamics


Thermodynamics is particularly interesting to sy
stems scientists for two major reasons.
First, it is the first place where complexity was recognized as a serious problem for
scientists. In addition, the concepts of a system were refined and solidified in the field.
This refinement was necessitated by

the problems faced by early thermodynamicists,
which were unique at the time. Of particular interest was the dual approach to
thermodynamics that utilized both top
-
down and bottom
-
up techniques to define
thermodynamic systems. Although mathematical syst
ems ideas had appeared in
Newtonian mechanics, thermodynamics could be considered to be the true birthplace of
mathematical study of systems.


Complexity first became a problem as scientists increased the resolution of their
instruments and probed more d
eeply into the natural world. It was observed that certain
systems did not obey the conservation of energy that was predicted by the simple
Newtonian models. Early efforts to explain this discrepancy made use of what are now
known as top
-
down techniques
and attempted to develop a theoretical model from macro
-
scale observations. While attempts by Carnot, Joule, and others to place this discrepancy
into a theoretical framework by adding additional interactions via materialistic methods
like caloric theory
[7] yielded little result, these efforts did result in a successful
axiomatic theory of thermodynamics.


The framework provided by the axiomatic theory defined the problem sufficiently
enough to allow Clausius to redefine the elemental component of the s
ystem. Clausius
proposed that heat was the product of the movement of a large number of molecules [7].
This theory implied that nature was significantly more complex than previously believed,
and that realization led directly to the heart of the problem
. It was known that the number
of molecules in even the smallest system was much too large to deal with given the
computational aids of the time period. Maxwell and Boltzmann dealt with this problem
by proposing that the molecules and interactions were h
ighly similar and that statistical
methods were applicable [7]. By grouping the molecules and interaction by property into
a relatively small number of categories, and by making use of the framework provided by
statistics and the creation of some syntheti
c concepts, such as entropy, to manipulate and
measure these categories, thermodynamicists were able to create relatively simple models
from which reliable predictions could be computed.


Both the axiomatic theories and the statistical theories proved valu
able to
thermodynamics, and both are still in use today. Gibbs was one of the primary
researchers of the period, and was active in both the axiomatic work and the molecular
theory. It was he who finally unified both in 1906 [8]. These two theories gave
systems
theory much of its vocabulary, including the concept of system boundaries, and sparked
later research. The importance of thermodynamics in the evolution of systems theory
cannot be minimized.


2.3 Ecology and Genetics


Early research in ecology
was done during the same approximate time period as early
research in thermodynamics. While thermodynamicists probed the intricacies of the
atomic world, social scientists and ecologists were faced with similar intricacies when
studying population dynamic
s. However, the problems faced by ecologists were not
identical to those found in thermodynamics and the differences, although small, were
significant. The habitats that ecologists studied contained a large number of diverse
animals. In addition, the fu
ll range of interaction among the fauna and flora in the habitat
were seldom known. These two obstacles translated into a system with a large number of
components and a large number of interactions, many of which were unknown.


The primary difference betw
een ecology and thermodynamics is that animals are
complex entities that are assumed to be somewhat consciously goal
-
oriented and
adaptable. Because of this, animals are nowhere near as constrained in the number of
interactions with each other and their h
abitat as are simple molecules of a gas. This leads
to a different and potentially more troubling type of system complexity. The numbers of
animals in most habitats were not so vast as the number of molecules in any
thermodynamic system. However, animal
s tend to move and hide where gas molecules
do not, which makes counting animals just as problematic as counting molecules. Even
though it is often assumed that animals of the same species have certain identical
characteristics and goals, the study of nat
ure has provided much evidence that animals
will often achieve the same goals in diverse ways. As a result of these differences,
ecological systems are complex systems not only because of the number of elements, but
because of the number of possible inter
actions between elements. For early ecologists,
these problems necessitated a slightly different approach than those of thermodynamics.


The statisticians Fischer [9] and Pearson [10] heavily promoted the use of statistical
methods, already successfully

applied to thermodynamics, in ecology. However, due to
the different nature of the problems, the applications of statistics were different.
Regression, chi
-
square testing, and concepts like standard deviations, were developed to
mask interaction complexi
ty [9], [10]. Sampling techniques came into widespread use in
ecology in order to measure the number of elements in the system. Verhulst's work on
the development of population equations [11] and the work of Volterra and Lotka [12] on
predator
-
prey dynam
ics made heavy use of both of these types of techniques.


Although genetics and evolutionary theory did not immediately introduce any systems
problems, they are worthy of mention both because they evolved directly from attempts
to apply ecology the concept
s of teleological causation to ecological systems and due to
the tremendous effect they have on modern biology. Beginning with von Goethe’s
morphology [12.5], genetics and evolution provided a new framework within which to
place ecology and pioneered the
use of stochastic modeling in biology with Mendel's
work on inheritance [13]. The importance of genetics and evolution would be very
important to biologists of the future.


3 The Quantum Revolution


The theory of quantum mechanics evolved directly out

of efforts by Planck and others to
solve the blackbody radiation problem from thermodynamics around the turn of the 20th
century [14]. Quantum theory heavily influenced the evolution of systems theory,
although the primary contribution of quantum revolut
ion to systems thinking was
philosophical. As thermodynamicists had previously questioned the possibility of a
simple universe, the new physicists questioned the possibility of a certain universe. The
success of quantum mechanics brought a new knowledge
-
oriented scientific paradigm
into the mainstream. It was from this paradigm that many of the ideas in formal systems
theory were born.


The ideas of nondeterminism, abstraction, and relativity, introduced into the scientific
mainstream by physicists, exer
ted a more significant influence on the philosophy of
science than on physics itself. The non
-
intuitive models of a quantum system and
relativity, as well as the resulting debates over the meaning of the results given by these
models, brought into questio
n the nature of human knowledge and introduced a new
knowledge
-
oriented, relativistic philosophy of science. This new philosophical approach
towards science heavily influenced the evolution of computational theory and the
development of the computer and w
as a major driving force behind the later development
of General Systems Theory and cybernetics.


Quantum theory is a difficult case to analyze in this context due to the on
-
going nature of
the debate over the theories’ physical significance. Although t
he Copenhagen
interpretation of the theory is favored by many, others would argue for a different
interpretation. Since the debate is not fully resolved, it is difficult to determine the
ultimate impact of quantum theory on science and on the philosophy o
f science. While it
is still too early to properly set this revolution into historical perspective, but it seems that
a significant shift of some sort has occurred in the way humanity views science as a result
of these theories.




Figure 3:

A Timel
ine of Key Events During the Quantum Era


3.1 Modern Physics


Modern physics is a difficult case in that the debates surrounding the interpretations of
the theories are not yet completely resolved. Thus, it cannot be said with certainty what
quantum theor
y and relativity's ultimate contributions to systems theory will be, but the
debates surrounding the theories have had immeasurable impact on both the philosophy
of science and the development of systems theory. Not only did these theories extend
existing

system concepts via nondeterministic methods, but the subsequent debates
helped introduce these methods into the mainstream. The debates over the interpretation
of the quantum model and relativity, as well as the debates over how far these new
theories c
ould be extended called into question the nature of scientific knowledge and
provided the modern philosophical context in which systems theory and systems biology
currently resides.


By 1900, physicists had discovered subatomic particles and had overturn
ed the idea that
the atom was the fundamental component of the universe. Early attempts by Bohr,
Planck, and others to reconcile the orbit of the electrons with Newtonian mechanics met
with limited success and brought up troubling inconsistancies [14]. T
o resolve these
inconsistencies, two competing but equivalent models were proposed in the 1920s. The
wave equation model introduced by Schrödinger and the algebraic model introduced
independently by Dirac and Heisenberg , while useful. were very different

from anything
previously imagined [15]. In these quantum models, observables existed in a state of
superposition and were assigned probability functions instead of values. The exact state
of the observable was unknown until the wave function, which enco
ded all possible states
of the observables, was collapsed. The concept of replacing a classical observable value
with a probability function was similar to the type of statistical manipulations found in
thermodynamics, but the way in which the nondetermin
ism was applied to a single
particle was unprecedented in the world of physics. The potential implications of the
quantum model started a series of debates that is still unresolved today.


Meanwhile, in addition to the new quantum theories under developme
nt, Einstein
proposed his theory of general relativity in 1916 [16]. In the relativistic model, time is
formalized as a fourth dimension and the philosophical concept of frames of reference
was applied to physical phenomena. The concepts of observers and

multiple frames of
reference were common in philosophy, but had not been applied formally in a scientific
theory at that time. The concept of relativity provided a twist to the debates raging over
quantum theory, being used by both Bohr and Einstein to s
upport their interpretations of
the quantum model. Later debates also arose over the extensibility of the relativistic
model; although a large amount of data supported the theory, concepts such as Gödel
loops [16.5] were as difficult to accept as superpos
ition.


The debate over the interpretation of these models, in particular quantum theory, has been
ongoing for the better part of a century. Bohr and his disciples championed the
Copenhagen interpretation, which proposed that the universe was actually
no
ndeterministic. Einstein and Schrödinger held that their interpretation, which proposed
an unknown deterministic process lay beneath quantum theory, was the correct one.
Feynmen, among others, claimed it was an unimportant detail. Regardless of which
ar
gument is correct, no one can deny the utility of quantum mechanics and relativity.
Likewise, no one can deny the immense impact the debate had on the philosophy of
science. Bohr’s argument brought up the unpleasant idea that humanity might never
possess

complete knowledge of the universe, destroying the mechanistic philosophy of
the preceding decades. Meanwhile, Einstein’s arguments focused attention on the
philosophy of science and brought the concept of relativism into the scientific
mainstream. The
nature of the Bohr
-
Einstein debate served to focus attention on the
proper role of philosophical relativism and of nondeterminism in science. Although the
Copenhagen interpretation is the currently favored theory among most physicists, there
are some who
would still disagree.


Likewise, the full impact of the quantum revolution on systems theory cannot be
measured. Nevertheless, the impact to date has been immense. These theories extended
system concepts already in use in other disciplines, and the nat
ure of the debates
surrounding them served to focus attention on the underlying assumptions behind such
concepts, leading to further refinement. The philosophical ramifications of quantum
theory and relativity, as well as the techniques developed within t
hese models, heavily
influenced the later development of systems theory and systems biology.


3.2 The Vienna Circle


The Vienna circle, a group of mathematicians and philosophers, was formed in the early
1920s under the guidance of Mortiz Schlick. In an a
ttempt to bring to philosophy the
rigor of scientific work, Schlick, Carnap, Neurath, and others developed a doctrine of
logical positivism based on Russell and Whitehead's mathematical logic [17.9]. Logical
positivism is a reductionist approach that stri
ves to verify statements by determining how
meaningful they are, or in other words, whether they can be verified logically or by
sensory observation [18]. Although criticized harshly by Popper [18.5], logical
positivism helped to formalize the conception
of a scientific theory. Because they dealt
with the nature of philosophical knowledge, logical positivism was naturally applied to
the nature scientific knowledge.


The Vienna circle drew much of its inspiration from the new ideas in logic and
philosophy
of the time and from the quantum debates. Through the Vienna circle, these
ideas and the philosophies behind them were transmitted to the founders of formal
systems theory, many of whom were products of or were familiar with the Vienna circle.
The result
ing agreement and disagreement with the philosophies advocated by the Vienna
Circle played an instrumental role in shaping formal systems theory.


3.3 Ludwig von Bertalanffy and Organismic Biology


Von Bertalanffy is considered by many to be the fathe
r of modern systems theory [18.9]
and should also be considered to be the father of modern systems biology. He was the
first to argue that biological organisms should be viewed as an integrated whole, a result
of his recognition that biological organisms
were both complex and dynamic. His two
earliest books on the subject, 1928's
Modern Theories of Development

[19] and 1932's
Theoretical Biology

[20], proposed that biological systems, which he termed “organismic
systems”, could be modeled as self
-
regulati
ng, open systems [20]. Von Bertalanffy also
proposed that biological organisms could be modeled as open thermodynamic systems
and were thus self
-
organizing systems. These systems were characterized by their ability
to interact with their environment and g
ain new emergent properties through evolution
[21].


Unfortunately, von Bertalanffy’s proposals were far ahead of their time and there was
only a rudimentary understanding of molecular basis of biology. While he attracted
attention and generated a burst o
f effort in the area, his theories were also criticized as
being pseudo
-
scientific and were compared to vitalism. Without an experimental basis to
support and test his theories, there was both little defense against these charges and little
progress made
in developing the theories themselves. As a result, von Bertalanffy’s
theories only generated interest in a few, most notably the physicist Schrödinger [21.5].


Despite the lack of progress in developing his biological theories, von Bertalanffy also
fou
nded the later General Systems Theory and his ideas profoundly influence modern
systems biology. Von Bertalanffy was a product of philosophy in general and of the
Vienna circle specifically. He was also an ecologist by training and was knowledgeable
of t
hermodynamics. His work reflects all of these influences. Much of his terminology
was borrowed from thermodynamics, his emphasis on interactions arises out of his
background in ecology, and his writings reference philosophy, notably Aristotle and
Hartman
n. Even ideas from the Vienna circle appear in organismic biology and later, in
General Systems Theory. Von Bertalanffy and the rest of the systems movement were
ardently opposed to reductionism as proposed by logical positivism, yet von Bertalanffy
too
k an interest in constructing a formal language to represent scientific theory and
eventually created general systems theory as that proposed language. He was a key
influence behind the philosophy that drove the evolution of cybernetics, and some of his
b
iological theories based on thermodynamic systems are the focus of research in modern
systems biology [21.6].


4 Cybernetics, General Systems Theory, and Computational Science


The new knowledge
-
oriented philosophy introduced by the quantum revolution led
to the
investigation of the nature of human knowledge, which increased the study of logic,
computation, and the process by which scientific models were created. These
investigations generated a collection of general ideas about the structure of scientific

models and the representation of knowledge. They also led to metaphors comparing
human processes with mechanical processes, which were rapidly expanded beyond the
original computational metaphor introduced by logicians.


The study of logic and computatio
n inspired the modern computer via the Turing
machine and the associated analogies to the human brain, while the expanded set of man
-
machine analogies came to be named cybernetics and influenced a number of fields
before fracturing and having a number of f
ragments absorbed by General Systems Theory
(GST) [21]. GST, which promised to provide a unifying language for the construction of
scientific models, did not gain much favor in the scientific community, although it did
benefit from the later influx of cyb
erneticians. Unfortunately, cybernetics, GST, and
early computational biology were beset by the same challenges that afflicted von
Bertalanffy’s original attempt at systems biology. The difficulties of dealing with large
amounts of data and computing la
rge and complex problems were not realized until
attempts were made to actually solve the problems. Although the molecular basis of
biology was better understood during these attempts than during von Bertalanffy’s first
attempt at an integrative theory of

biology, the techniques required for data collection and
the mechanisms needed for data integration and computation on an appropriate scale
were still not available, nor were the difficulties of complexity completely understood.


Despite the challenges,

cybernetics and GST were the first attempts at developing a
systematic theory of biology and formalized the concepts of a system. Both fields have
left a rich legacy. Artificial intelligence, the behavioral and cognitive schools
psychology, Simon's work

in decision making in economics, systems analysis,
neurobiology, and physiology all bear the imprint of these fields, although the connection
is often not recognized. Cybernetics and GST have heavily influenced modern science
and modern systems biology,
despite their historical difficulties.




Figure 4:
A Timeline of Key Events During the Systems Era


4.1 Cybernetics


Cybernetics, the study of self
-
regulating and adaptive systems, can be viewed as the
second attempt at formulating a systematic theory of

biology. Although cybernetics was
an interdisciplinary field from the beginning, its birth can be traced to the earliest study of
computation. In 1943, the neurobiologists McCullough and Pitts published an article that
proposed a theory of how ideas ari
se from the activity of neurons in the human brain
[22]. This article, along with an article by Rosenblueth, Wiener, and Bigelow [23] on the
philosophical issues of teleology and purpose, marked the birth of cybernetics. From the
limited analogy between
the human brain and early computers grew a host of analogies
between humans and machines drawn from a diverse interdisciplinary background. This
collection of analogies evolved into a generalized study of self
-
regulating systems.
Feedback and control theo
ry were borrowed from engineering to describe self
-
regulating
processes in biology, information theory was borrowed from electrical engineering and
used to describe the communication process required for control, system ideas and
terminology were imported
from physics and thermodynamics, and many other ideas
from diverse fields such as linguistics and biology also found a home in the discipline.


In Wiener’s
Cybernetics: Communication and Control in the Animal and Machine
,
published in 1948 [24], Wiener

also credits von Bertalanffy’s organismic biology,
information theory, and neurology as the foundations of cybernetics. Wiener’s outlook
on cybernetics, as a fundamentally nondeterministic field interested more in interactions
between things than things
themselves, was shaped by his wartime experience in directed
anti
-
aircraft fire, which used stochastic modeling in an attempt to predict the path of an
airplane weaving randomly in an attempt to avoid antiaircraft fire [36]. Cybernetics,
particularly Wien
er’s view of cybernetics, achieved significant popularity among
biologists. Included in this list were the neurobiologists McCullough and Pitts, as well as
Ashby, who wrote an excellent book covering most of the field from the perspective of a
biologist [
25].


Although Wiener, Ashby, von Foerster, and others argued for the need for stochastic
mechanisms in cybernetics, many cyberneticians came from an engineering background
and were more familiar with engineering problems and engineering problem
-
solving
t
echniques. As a result, cybernetics was infused with many deterministic techniques, and
made use of differential equations and mechanistic engineering concepts such as
feedback loops and Shannon’s information theory, which dealt with the effects of signal

and noise on communication channels and was a product of electrical engineering [26].
This view of cybernetics, while useful to problems of a manageable size and defined
purpose, had the side effect of fostering a 'culture of determinism' within cybernet
ics.
When more complex problems were encountered, this culture provided resistance to the
adoption of nondeterministic techniques and contributed to the fracturing of cybernetics.
Since the deterministic methods proved to be sufficient for the needs of m
any
cyberneticists, they returned to their own disciplines and abandoned what were
considered to be at the time irrelevant problems. Cybernetics experienced the problem
from which many interdisciplinary projects suffer, that is the transitory nature of th
e
participants’ interests, and fractured as researchers returned to their original disciplines.


4.2 General Systems Theory


General Systems Theory evolved out of von Bertalanffy’s background in the Vienna
circle around the same time period as the populari
ty of cybernetics peaked. General
systems theory was intended to form a formal language in which scientific models could
be constructed, despite attacks by its critics that it was attempting to be a universal
scientific theory itself. In fact, there were

universal elements to the theory and it was
abused in that form, but von Bertalanffy never billed it as any more than it was. GST was
primarily concerned with properties of and organization of systems in general, as
opposed specific collection of analogi
es. Von Bertalanffy considered cybernetics merely
a mechanistic form of general systems, and this view came to be shared by some of those
involved with cybernetics as they encountered more complex problems that could not be
solved by mechanistic means.


T
he principle characteristics of GST included the study of isomorphisms between
systems that would lead to general theoretical laws about systems, an emphasis on the
organization and function of systems, an emphasis on hierarchy within systems, and the
stud
y of emergent properties, which are behaviors that arise from the interactions among
individual parts [21]. Cybernetics quickly began to parallel general systems theory,
beginning with von Foerster’s work in second
-
order cybernetics [27]. The urging of
W
iener and Ashby for more use of nondeterministic methods and the search for motifs
common to cybernetic systems also paralleled common themes in general systems theory.
Upon the fracturing of the field of cybernetics, this parallel work was absorbed into
GST.
Despite the absorption of cybernetics, general systems theory was never affected by the
culture of determinism present in early cybernetics because most of those who migrated
to the field were those who required nondeterministic techniques to solve t
heir problems.
After the absorption of cybernetics, work continued in general systems by those like
Simon [27.5], Weinberg [28] and Klir [29].


The primary charge leveled at GST by critics is understandable in hindsight. GST was
more focused on a theore
tical level and many disciplines had acquired their deterministic
metaphors from cybernetics by the time of absorption. These fields found the emphasis
on theory and the move away from a mechanistic view to be less than useful; certainly
neither the behav
ioral school of psychology or engineering control theory had any use for
a non
-
mechanistic view at the time. The problems those two areas dealt with were
generally not recognized as complex enough to require much in the way of
nondeterminism. As a result
, this drift into nondeterministic methods along with GST’s
more pronounced penchant for finding isomorphisms were decried by many as a sign of a
lack of relevance to real world applications.


Russian cybernetics, or what would more appropriately be called

Russian General
Systems Theory, is a more interesting case than the Western applications of GST. In the
Soviet Union, cybernetics was accepted as a unified language of science, in the role that
general systems theory was proposed to fill, and most scient
ific research took place in
this context [29.1]. This was particularly interesting in its effect on biological research.
Despite government repression of genetic research, Liapunov had proposed ideas bearing
a striking resemblance to those in modern syst
ems biology as early as 1960, including the
need for high
-
throughput data collection [29.2], an information theoretic study of the
transmission of genetic information [29.3], and hierarchy of biological control systems,
with each system functioning as an e
lement of a higher level system [29.4]. The
Russians were also more accepting of stochastic methods, perhaps because Markov and
Kolmogorov had pioneered many stochastic modeling techniques.


Unfortunately, GST failed to progress much further than cybernet
ics or organismic
biology because the field was still grappling with the same problems of complexity that
Systems Biology is faced with today. To compound the difficulty, these problems were
more pronounced during GST's infancy than they are today. Compu
ting technology was
much more primitive, large amounts of biological data were unavailable, and
computational complexity was not fully understood at the time. Unlike the earlier
attempts at formulating an integrated theory of biology, GST has been able to

adapt and
absorb new concepts like fuzzy sets [34] and stochastic modeling. Despite the challenges
faced by GST, it has persisted and exerts a direct and continuing influence on modern
systems biology [34.1].


4.3 Computational Science and Early Computat
ional Biology


Computational complexity has always accompanied attempts to compute scientific
problems, even if the phenomenon was not understood or recognized immediately due to
other limitations. The earliest attempts at computational biology were not d
irectly
hindered by complexity, but instead by the available hardware of the time and by limited
data on biochemical processes. The first work in computational biology was Tarski and
J.H. Woodger’s work on axiomatic biology [30], which was an attempt to r
epresent
biological knowledge as a logical system in which biological truths could be found. This
early work paralleled the research of Turing, Church, and Godel in the 1930s and was
spurred by the general study of human knowledge and logic during that ti
me period.
Upon creation of the first electronic computers, physicists, chemists, and biologists alike
quickly recognized the potential of the new machines. Physicists and chemists put these
earliest computers to work on their problems with success, and
several useful
biochemical simulation packages and statistical analysis packages were also developed.
Simulation software, such as Biobell [31], which combined nonlinear differential
equations with discrete event simulation, was the norm throughout the ne
xt three decades.
These programs neatly illustrate the immense problems with which early computational
biology was faced. In consequence, the vast majority of these programs were designed
for small
-
scale simulation of a sequence of reactions. The few at
tempts at larger scale
simulations, like metabolic control theory and biochemical systems theory [32], were
only a matter of degree.


Attempts at modeling biological systems on a large scale foundered for two major
reasons. First, the hardware of the time

was not capable of handling the complex
nonlinear differential equations that were the hallmark of biological modeling. In
addition, constructing these large systems of equations was hindered due to incompletely
specified interactions between the biochem
ical species. This imposed severe limitations
on what could be done by computational scientists and biologists. Second, the very
nature of some problems made them difficult to compute. This was not a problem limited
to only biologists; it appeared throu
ghout computational science and led to the eventual
investigation of computational complexity.


The formal study of computational complexity began in the early 1960s [32.1] and dealt
with the resource cost, in space and time, of solving classes of problems
. However, it did
not receive much attention from the computational science community at large until Cook
[32.2] and Karp’s [32.3] work on NP problems in the 1970s. Complexity theory splits
problems in hierarchical differences based on fundamental proper
ties of the problem, like
interaction level, and shows that some problems are more difficult to solve than others. It
also links the resource requirement to the problem’s input size. An understanding of
complexity theory gives an understanding of why bio
logists have such difficulty in
dealing with large amounts of biological data and constructing and computing biological
models. Likewise, an understanding of the difficulty of computing some biological
problems emphasizes the points made by complexity the
ory.


5 Nondeterminism and Chaos


In the late 1960s and early 1970s, work in computational complexity demonstrated that
some problems were either not computable by current techniques or were extremely
difficult to compute. The ramifications of complexit
y theory ended the last remaining
shreds of hope that the world could be viewed deterministically or certainly. A host of
techniques were adopted in an attempt to work around the difficult problems, including
fuzzy logic, stochastic systems, and chaos the
ory. Systems biologists have adopted many
of these techniques, although this process is not yet complete and no conclusions can be
drawn from their uses. In many cases, under what circumstances and how these
techniques might be applied to biology are not

yet fully defined. Nevertheless, they have
been applied with some success and are worthy of further investigation, so a short
overview of their uses in modern systems biology is given here.


5.1 Fuzzy Computation


Fuzzy logic is merely a form of logical
system where variables can take on more than the
standard two values. Fuzzy computation is the use of fuzzy logic to solve problems.
Fuzzy logic was developed in the 19
th

century, but it was not until Zadeh’s paper
Fuzzy
Sets
, published in 1965 [33] that

any serious interest was taken. Even then, it was largely
ignored until researchers in artificial intelligence and general systems discovered it a few
years later. Klir is one of the more important figures in fuzzy systems and has authored a
book on the

subject,
Fuzzy Sets and Logic

[34].


Long used in engineering and control theory, fuzzy computation has recently been used
in the analysis of genetic data obtained from micro array experiments, where
comparisons can be more qualitative than quantitative.

Fuzzy computation has also been
used to deliberately fuzz variable values in order to allow use of a simpler model. There
are commercially available clustering algorithms that make use of fuzzy logic, such as the
FANNY algorithm [34.1]. In general thoug
h, there have been few applications of fuzzy
logic in biology, as there are problems involved in the use of fuzzy computation, not the
least of which is the difficulty in defining the member functions. Nevertheless, due to the
qualitative nature of some o
f the comparisons given by micro array technology, fuzzy
logic does seem applicable in some cases and has been applied with some success.


5.2 Stochastic Systems


Stochastic techniques are ancient and can trace their foundations at least back to early
th
ermodynamics. However, stochastic techniques were not widely adopted by scientists
until much later, after some of the philosophical issues of science had been laid to rest.
The Russian mathematician Markov constructed the probabilistic paths that bear h
is
name as early the 1930s and some early cyberneticists readily adopted stochastic control
circuits. In fact, many of the early ideas of cybernetics can be found in Wiener’s wartime
work on controlling and predicting anti
-
aircraft fire [36] and his earli
er work on
Brownian motion [36], both of which used stochastic techniques to model what seemed
to be random phenomena.


Stochastic techniques gained early acceptance in the West for use in biology and physics,
and were also heavily used in the Soviet Un
ion in biochemical modeling. Adoption of
these methods lagged in the engineering disciplines, partly due to advantages inherent in
engineering problems that reduced the need. However, with the increasing complexity of
modern technology, there has been in
creased interest in stochastic modeling in
engineering. Currently, there are a number of biochemical modeling software packages
that make use of stochastic modeling. Stochastic techniques are also often used in the
analysis of genetic data. The wide sco
pe of application prevents a comprehensive survey
in this venue; however, this widespread application and increased interest tends to speak
for the potential utility of the methods.


5.3 Chaos Theory


The first research into chaos theory grew out of early
efforts in weather prediction,
although Poincare first noticed the phenomenon in the three
-
body problem around 1900
[36.6]. In 1960, Lorenz noticed that weather prediction models diverged wildly when
very small changes in input parameters were made. Afte
r further exploration, chaos
theory became an integral part of stability analysis and is currently in heavy use in any
field dealing with nonlinear systems. In biology, where chaotic systems were first
noticed in ecology [36.7], is among these fields.


Ch
aos theory is currently used heavily in biological modeling. It has found a role in
modeling emergent properties in nonlinear systems of equations and modeling the
stability of large systems of nonlinear differential equations. Of particular note, it has

been proposed that changes in the chaotic nature of a biological system affect the fitness
of the system [37]. This technique has been used to diagnose electrocardiogram data and
predict heart failure [37]. Chaos has also been used to model data encodin
g in DNA
molecules [37.2]. In some circles, chaos theory has attained the reputation of being over
-
exposed, but there is enough applicability to biology that it should not be ignored.


6 Balance Sheet and Historical Conclusions


Currently, biology is a fi
eld undergoing rapid change and development. Computer
technology, high
-
throughput instruments like the micro array, and greater knowledge
about small
-
scale biological processes, have given biologists large quantities of data as
well as the techniques to m
anipulate this data. However, as more data becomes available,
more evidence demonstrates the complexity of any biological phenomena. This
complexity poses many problems for the modern biologist that cannot be overcome by
traditional methods.


The histo
rical evolution of systems ideas shows a constant struggle against complexity.
Thus, the role of complexity in modern systems biology should not be ignored. Modern
biology has developed several different categories of techniques for dealing with various
facets of the complexity inherent in biological systems and the problems this complexity
generates. All of these categories are loosely grouped under varying names, such as
Bioinformatics or Systems Biology. In general, Bioinformatics deals with the coll
ection,
management, and analysis of biological data. Systems biology, mathematical biology, or
computational biology, deals with
in silico

modeling and the mathematics of biological
systems. The interconnected nature of biological phenomena and the need
for each of
these categories makes distinctions between the terms rather small.


[Coming Soon, its a Venn Diagram of sorts...Left it at home]

Figure 5:

What is Systems Biology?

From Kitano, "Systems Biology: A Brief Overview" [37.5]


6.1 The Role of Bioi
nformatics


The importance of modern data collection techniques in genetics and molecular biology
cannot be minimized. Although the role of incomplete information has been minimized
through much of this paper, it is as serious a problem as modeling biolog
ical systems.
Both of these problems stem from the complexity of biological systems. One difference
between the current efforts and past efforts is that until the latter half of the 20
th

century,
biochemistry and genetics were not well understood. Thus,

there was a lag in developing
the techniques needed to gather this data.


The micro array is arguably one of the most important innovations in biology in recent
times. The gene chip technology allows side
-
by
-
side comparison of a vast number of
data point
s. While the precision of the chips and thus their analytic capability is still
limited, it does allow collection of enough data to make large
-
scale biological analysis
possible. This is something that would take many years if done in the traditional way

and
was one of the primary problems with previous attempts at formulating a systematic
theory of biology.


Yet high
-
throughput data collection is only half of the solution. Once biological data is
gathered, it must be analyzed and put to use. A wide var
iety of statistical techniques and
some fuzzy logic have been used to these ends. Also significant is the use of
standardized XML languages and databasing techniques to manage the information and
promote dissemination and portability among researchers. S
BML [39] and CellML [40]
are notable languages for the storage of biological models, while there are quite a few
databases for protein and genetic information ranging from the small to the enormous.


6.2 Current Work in Systems Biology


To say that systems

biology is a young field is an understatement at best. However, the
fundamental concepts are not. As the historical evolution of systems theory displays, the
concepts have been evolving for centuries. Nevertheless, without this fortuitous
intersection
of readily available biological data and the results of the centuries long
evolution of mathematical techniques designed to deal with this type of complexity, a
systems approach to genetics and biochemistry would be impossible.


A report published by the
World Technology Evaluation Center (WTEC) details current
work in Systems Biology [41]. The definition of Systems Biology given by this report
shows a discipline driven by emphasis on the dynamic properties of biological networks.
There are a wide variet
y of techniques in use, including nonlinear differential equations,
Bayesian networks, Petri
-
nets, cybernetic motifs, and optimization
-
based thermodynamic
models. Work in systems biology seems to be centered in the US, Japan, and Western
Europe. Two cen
ters for the study of systems biology have been established; one is
located in Japan [42] and the other is located in the U.S [43]. Many more programs have
been opened recently or are planned. Systems Biology, while young, is growing rapidly.


6.3 System
s Biology and Computational Complexity


Systems biology is the product of centuries of evolution in systems ideas. Although a
direct link between systems biology and general systems theory is evident, systems
biology can also benefit from lessons learned
in thermodynamics and ecology. Problems
in modern biology share elements from problems in both thermodynamics and ecology.
There are a large number of elements in a biological system, and these elements appear to
be complex and highly interactive. Therm
odynamics and ecology have achieved
different levels of success in dealing with these problems; it remains to be seen if systems
biology can successfully deal with both. It should be noted that cybernetics foundered on
this type of complexity and the same

fate almost befell general systems theory. Modern
mathematical and computational techniques have vastly improved since, but it is as of yet
unclear on how to use them properly.


For better of for worse, it seems likely that the future of systems biology

is tied to
complexity. Biologists might do well to take the philosophical lessons learned by
physicists and engineers to heart and focus more on problem characteristics and model
constraints rather than searching for universal correctness. In this appro
ach,
computational science might be of some assistance. Computational science has already
assisted biologists with this complexity in managing and collecting biological data. As
computer science has a long history in dealing with complexity, although adm
ittedly with
limited success, a partnership on an even higher level than is currently common would be
productive for both fields. Many biologists lack training in complexity and many
computer scientists lack training in biology. However, computational sc
ience, with its
emphasis on problem analysis, could offer assistance to biologists seeking to avoid
difficulties associated with complexity. Likewise, three emerging trends in computer
science are biologically inspired computing, fault tolerance, and self
-
organizing systems;
an understanding of biological systems would greatly benefit both of these trends, as
biological systems are among the most complex known yet seem to make use of these
principles fairly well. One ironic example along this line of thou
gh is the employment of
genetic algorithms, originally inspired by evolutionary concepts, to analyze data from
genetic experiments.


More immediately, computational issues limit the amount of data biologists can integrate
and study as a whole. Biological
systems are composed of a large number of diverse
elements which interact with each other in many, diverse ways. Although it is currently
unknown how closely biological systems approach the upper boundary of system
complexity of 2
n
, established by Weinber
g [28], it seems certain that biological systems
are closer than many other systems. This complexity both makes high
-
throughput data
collection necessary and limits its utility at the same time.


In the long term, biological systems might be extremely dif
ficult or impossible to model
deterministically. Proteomics is still a young science and the definition of what
constitutes an element in a biological system is still in doubt. It may be the case that
biologists are faced with hard problems nested inside

hard problems or that some
problems are not computable in the Turing model of computation. There is also no
guarantee that computers can handle the complex differential equations to the needed
degree of accuracy, especially since biological systems seem
to be chaotic in some sense.
Computer technology advances rapidly, but there are theoretical limits or difficulties that
cannot necessarily be overcome easily.


Despite these potential problems facing the field, systems biology is uniquely positioned
to
provide some answers to these problems. It seems likely that once properly matured,
the field of systems biology will yield useful results in multiple fields, not merely in
biology. Systems biology and computational science both evolved from the same
bac
kground and both fields face the same issues of complexity and computability. The
historical evolution of the shared system ideas paints a picture of a struggle against
complexity. The difference is that with evolving mathematical techniques, increased
u
nderstanding of biochemistry and genetics, and high
-
throughput data collection,
progress in systems biology seems inevitable.


7.0 References


[1] Lloyd, G. E. R.
Early Greek Science: Thales to Aristotle
. Norton, New York, 1970.

[2] Osborne, Henry Taylo
r.
Greek Biology and Medicine
. Cooper Square Publishers,


New York, 1922.

[3] Newton, Issac.
The Mathematical Papers of Isaac Newton
. Derek T. Whiteside, ed.
1967.

[4] Descartes, Rene.
The Philosophical Writings of Descartes,

Vol. 1. Cottingh
am, J,
Stoothoff, R., and Murdoch, D., eds. Cambridge University Press, 1985.

[5] Pahl, G.
Engineering Design: A Systematic Approach, 2
nd

ed.

Springer, 1999.

[6] Engleberg, S.
A Mathematical Introduction to Control Theory.

WSPC, 2005.

[7] Keenan, J.
H. and Hatsopoulos, G.
Principles of General Thermodynamics
. Pg xv
-
xli,
1965

[8] Gibbs, J.W.
The Collected Works
, Vol. 1. 1948.

[9]
Fisher, R. A.
The Genetical Theory of Natural Selection
. Clarendon, 1930.


[10] Pearson, E.S (ed.).
Karl Pearson’
s Early Statistical Papers
.
Cambridge University
Press, 1948.

[11] P.

Verhulst.
Recherches mathématiques sur la loi d'accroissement de la population
.

Nouveaux mémoires de l'académie royale des sciences et belles
-
lettres de Bruxelles
,
18:1
-
38, 1838.

[1
2] Lotka, A.J.
Elements of Physical Biology
. Williams and Wilkins, 1925.

[13] Bateson, W.
Mendel's Principles of Heredity, a Defense
, First Edition, London:
Cambridge University Press, 1902.

[14] Bohr, N.
The Philosophical Writings of Niels Bohr
, Vo
l. 1. Ox Bow Press, 1987.

[15] Jammer, Max.
The Conceptual Development of Quantum Mechanics
. Wiley, 1974.

[16] Einstein, A.
Relativity: The Special and the General Theory
. Henry Holt, 1920.

[19] Bertalanffy, L. von.
Modern Theories of Development:

An Introduction to
Theoretical Biology
.
Oxford, 1933. (translated).

[20] Bertalanffy, L. von.
Theoretische Biologie
. 2 Bde., Berlin, 1932.

[21] Von Bertalanffy, L.
General Systems Theory: Foundation, Development,
Applications
. George Braziller, 1
976.

[22] McCullouch, Warren, and Pitts, Walter. “A Logical Calculus of the Ideas
Immanent in Nervous Activity”, Bulletin of Mathematical Biophysics, Vol. 5 (115
-
133).
1943.

[23]
Rosenblueth, A., Wiener, Norbert, and Bigelow, Julian. “Behavior, Purpos
e, and
Teleology.” Philosophy of Science, Vol. 10 (18
-
24). 1943.

[24] Wiener, N.
Cybernetics: Control and Communication in the Animal and the
Machine
. MIT Press, 1965.

[25] Ashby, R.
Introduction to Cybernetics
. Routledge, Kegan, and Paul, 1964.

[2
6] Shannon, Claude E., and Weaver, Warren.
The Mathematical Theory of
Communication
. University of Illinois Press, 1963.

[27] Von Foerster, H.
Understanding Understanding: An Epistemology of Second Order
Concepts
.
Aprendizagem/Desenvolvimento
. Pg 83
-
85, 1981.

[28] Weinberg, G.
An Introduction to General Systems Thinking
. Dorset House
Publishing, 2001.

[29] Klir, G.
An Approach to General Systems Theory
. Van Nostrand Reinhold Co.,
1969.

[30] Woodger, J.H.
The Axiomatic Method in Biology.

Cambr
idge Press, 1937.

[31] Stevenson, D.E., Warner, D. D., and Brown, T.R.
Biobell: A Simulation System for
Biochemistry and Biophysics
.
Comput. Biol. Med.
, Vol. 14, No. 1. Pg 35
-
46, 1984.

[32] Savageau, et al.
Biochemical Systems Theory and Metabolic Co
ntrol Theory:
Fundamental Similarities and Differences
. 1987.

[33] Zadeh, L.A.
Fuzzy Sets
. Information and Control, June 1965. Pg. 338
-
353.

[34] Klir, G., and Yuan, B.
Fuzzy Sets and Fuzzy Logic: Theory and Application
.
Prentice Hall, 1995.

[36]

Mansani, P.
Norbert Wiener: Collected Works
. MIT Press, 1990.

[37] Dention, T.A., et al.
Fascinating Rhythm: A Primer on Chaos Theory and Its
Application to Cardiology.

Am Heart J, 120, 1419
-
1440 (1990).

[39] SBML website. www.sbml.org. Accessed 0
8/30/2005.

[40] CellML website. www.cellml.org. Accessed 08/30/2005

[41] WTEC Report on Systems Biology. World Technology Evaluation Center, Oct.
2005.

[42] The Systems Biology Institue. Tokyo, Japan. www.sbi.jp.

[43] Institute for Systems Biology
. Seattle, WA, USA. www.systemsbiology.org.


[16.5] Godel, K. "An Example of A New Type of Cosmological Solution of Einstein's
Field Equations of Gravity"
Rev. Mod. Phys.
21, 447, 1949

[29.1] Gerovitch, S.
From Newspeak to Cyberspeak: A History of Sovi
et Cybernetics
.
MIT Press, 2002.

[29.2] Sobolev, Sergei L. "Vystuplenie na soveshchanii" in
Philosophical Problems
, ed
Fedoseev, p 266 (From Gerovitch). Working on a better reference.

[29.3] Liapunov and Iablonksi. “Theoretical Problems in Cybernetics
.”

[29.4] Liapunov. "Ob upravliaiushchikh sistemakh" (From Gerovitch)

[32.1] Hartmanis and Steans. “On the computational complexity of algorithms” 1965.

[32.2] Cook, S.
The Complexity of Theorum Proving Procedures
. Proceedings of the
third annual ACM

symposium on theory of computing, 151
-
158.

[32.3] Karp, R. "Reducibility Among Complex Problems."

[34.1] L. Kaufmann and P. J. Rousseeuw.
Finding Groups in Data: An Introduction to
Cluster Analysis
. Wiley and Sons, 1990.


[0.5] Veith, Ilza, ed.
The
Yellow Emperor’s Classic of Internal Medicine
. University of
California Press, 1966.

[21.5] Schrödinger, E.
What Is Life?

Cambridge University Press, 1944.

[21.6] Schneider, ED, and Kay, J.J. “Life as a Manifestation of the Second Law of
Thermodynamics
.”
Mathematical and Computer Modeling
. Vol. 19, No. 6
-
8, pg. 25
-
48.

[27.5] Simon, H. A.
The Sciences of the Artificial, 2
nd

Ed.

MIT Press, 1981.

[36.6] Barrow
-
Green, J.
Poincare and the Three Body Problem
. American Mathematical
Society, Novermber

1996.

[36.7] May, R. “When two and two do not make four: nonlinear phenomena in ecology.”
Proceedings of the Royal Society
, 1986, Vol. B228, pg. 241.

[37.2] Coffey, D. “Self
-
organization, Complexity, and Chaos: The New Biology for
Medicine.”
Nature
Medicine
,
4
, pg. 882
-
885, 1998.

[37.5] Kitano, H. "Systems Biology: A Brief Overview."
Science, Vol 295.

March 1,
2002.


[17.9] Russell, B. and Whitehead, A.
Principia Mathematica
. Cambridge University
Press, 1903.

[18.9] Davidson, M.
Uncommon Sense:

The Life and Thought of Ludwig von Bertalanffy,
Father of General Systems Theory
. 1983.

[34.1] Wolkenhaur, O. "Systems Biology: The Reincarnation of Systems Theory
Applied in Biology?"
Briefings in Bioinfomatics
. 2001.