High End Computing Terascale Resources (HECToR) Scientific Case 7 April 2004


Feb 22, 2014 (3 years and 5 months ago)


High End Computing Terascale Resources


Scientific Case

7 April


(Unrestricted version, dated 26 April 2004)




Executive Summary

and Context




Trends in Computational Science and Engineering


Scientific Val
ue of a Computing Infrastructure


Selected Applications


Computer Solutions


An Outline Roadmap


Bibliography and Acknowledgements


Detailed Applications Evidence

Abstracts of Research Projects using CSAR and HPC

Current Infra
structure and Research Base

D. DEFRA, EPSRC, the Met Office and NERC Partnering Letter



Executive Summary

and Context

{Removed since it contains restricted information, Hugh
Clayton, 26/04/04}




Computation has now become esse
ntial for the advancement of all research across
science and engineering. Access to a rich portfolio of applications software, and to
hardware facilities on which to run it, is indispensable to any competitive research

In all developed nations

an ecology of resources now exists, ranging from the desktop
through the department to the national levels and encompassing both centralised and
based distributed arrangements.

Within this ecology, High
End Computing

HEC for short

occupies a str
position. It aims to satisfy the most demanding scientific goals, to push the boundaries
of researchers’ ambitions and to stretch the development of hardware and software
technologies in dimensions that often prove beneficial outside the HEC researc
arena. For this reason, and because of its high costs, funding agencies tend to treat
HEC provision in a strategic way. Thus,

“Due to its impact on a wide range of federal agency missions … high
end computing
capability is becoming increasingly critical
. … Research and software to support
end computing will provide a foundation for future federal R&D …” US
Federal Budget FY2003.

An analogous statement would surely apply in the UK, where the Research Councils
have acted in a co
ordinated way to pro
vide and support a wide range of
computational resources. This has been one of the key factors in nurturing what is
now a high quality UK computational science and engineering research community
that is world
class or world
leading in many fields. The othe
r key factor has been the
UK’s collaborative approach to scientific software development and support: large
sectors of the UK research community have found ways of working together to
develop and exploit applications codes of a scale beyond the realistic a
spirations of
any individual group. It can certainly be argued that in this way the UK gets more
science from its investments in computational resources than most nations.

At the time of writing, mid 2003, the UK has very competitive HEC facilities
ble to its researchers, in the form of the CSAR and HPC

services: HPC
is still
the second most powerful academic HEC facility in the world (according to the Top
500 list published in June 2003 [1]). It is certain that this will not be the case for very
much longer. In the USA, the Extended Terascale Facility is well underway, coupling
together several large commodity cluster, special purpose HEC and data systems
within a high bandwidth Grid [2]. The NSF’s next HEC initiative

the Cyber
Infrastructure pr

is expected to begin towards the end of 2003 [3]. In Japan
the Earth System Simulator (ESS)

a very large parallel system in which each node is
itself a powerful vector processor

has created enormous interest in the HEC world
[4]. Development
of the Numerical Simulator at Japan's National Aerospace
Laboratory (NAL) provides further evidence of Japan's investment in Terascale


facilities [5]. There are several intriguing novel low
power architectures exemplified
by IBM’s Blue Gene family currentl
y under development [6]. As a matter of fact, this
is a more promising period than the HEC arena has enjoyed for a number of years. As
Horst Simon, Director of NERSC, says “The world of supercomputing is churning,
and something new will come out.” [7].

t is therefore essential that we proceed now with the acquisition of a new generation
of HEC provision as envisaged in RCUK’s Large Facilities Roadmap. Such provision
is strongly endorsed within HSC's "A Strategic Framework for High
End Computing"

For the UK to enhance its position amongst the world
leaders in computational science
and engineering, there must be significant investment in the development of skills, and
researchers must be provided with access to high performance computers that are
ongst the best in the world

with regular updates to keep pace with the accelerating
rate of technological advance" (July 2003)

“Roadmaps” are indeed the appropriate planning concept for HEC strategy
development. The HSC is currently developing roadmaps
for both hardware
technologies and applications software. The key observations emerging from these
exercises are:

that clear and specific scientific and engineering research goals can be
reached given increases of around 100
fold in computational performa

that such increases in theoretical performance are within the technology

that the applications software roadmap can also encompass the developments
in functionality and delivered performance needed to reach
these goals

Following an initia
l overview of trends in computational scie
nce and engineering in
Section 3
, we describe in Se
ctions 4 and 5

the computational aspects of a wide range
of science a
nd engineering fields. Section 4

considers a number of strategic criteria
pointing to the cruc
ial role of computing infrastructure provision across the entire
spectrum of scientific applications. We focus on the six criteria mentioned in the
Large Facilities Roadmap

the importance of HEC to research in the scientific fields,
the breadth of scienc
e involved across these fields, the position of UK HEC
activities internationally, the opportunities for training provided through HEC
provision, the contributions to and benefits from UK industry in HEC, and finally the
opportunities for spin
and exploitation presented through infrastructure provision.
Addressing sixteen fields of scientific endeavour, Section 4 outlines the science goals
and computational requirements of each field in turn

these are based on more
detailed applications eviden
ce (availa
ble at Appendix A). In Section 6

we turn to a
consideration of the likely computer solutions, outlining the range of available and
announced high
end architectures

custom and commodity clusters, vector systems
and "systems
chip" architectu
res. Section 7

outlines a roadmap linking key
applications to future technology advances, and addresses the science that can be
reached given different levels of computational performance.



Trends in Computational Science and Engineering

The following sect
ions of this paper describe the computational aspects of a wide
range of science and engineering fields. Each has its set of science goals, software
characteristics and, following on from that, hardware requirements. For each field, we
indicate the science

that can be reached given different levels of computational
performance, and outline the match to RCUK’s strategic criteria. It may be useful to
draw out from this analysis a few general messages.

First, we observe no slackening of the pace at which the
scope and impact of HEC
research is increasing. Fields traditionally associated with HEC are continuing to
flourish, while new communities (especially in the life sciences) are making more and
more use of HEC. In a report written in 2001, TOP noted a stron
g trend towards the
modelling of whole systems as well as system components [9]. This continues and
strengthens, with progress being made in materials science (especially nano
bioscience and engineering. Indeed, a common theme that runs through m
any of the
scientific areas under discussion is the integration of several computational techniques
to attack problems with a number of different length (and perhaps time) scales. Such
models will be

and therefore

whole syste
ms generally
involve a wide range of both length and time scales. Biology, for example, in
spanning phenomena at the atomic, biochemical, cellular and organism levels, covers
about 10 decades in length scale (and about 24 decades in time scale, if evolutio
n is
included). Systematic methods of breaking these scales into manageable but
meaningful chunks are under investigation everywhere. It is also clear that a number
of fascinating new areas could be opened up by access to
, real
modelling being a prominent example.

Second, as always, software is the key to achieving science and engineering goals: the
application software embodies the science, in fact. Software development trends
include the need to build and maintain complex appl
ications, with single
monolithic codes giving way to packages assembled by multi
disciplinary teams. This
brings an additional need for software engineering and quality disciplines. These are
most easily accepted in the context of collaborative deve
lopment of community codes,
and the UK is well placed to take advantage of this. Many fields are expecting a data
intensive as well as compute
intensive era, and looking towards Grid
developments to address this. Probably the most pressing softwa
re challenge is the
increasing difficulty of achieving high, sustained performance in complex
applications. This has much to do with processor
memory bandwidth and latency,
with inter
processor bandwidth and latency, and with the trade
offs hardware
ufacturers have to make in building large systems to a price. These problems
impact science goals most often in the crucial dimension of scaling, i.e. the efficiency
with which additional processors can be used to achieve additional performance.
Today, man
y applications codes can effectively exploit around 1000 processors
(normally after considerable effort in tuning and optimisation). Raising this limit by
one or more factors of 10 is an active area of R&D, and is likely to remain so, as
system architectur
es evolve and emerge. A different and perhaps even more
fundamental issue is that time is a difficult variable to parallelise because of its role in
causality: sequential causes imply sequential programs. This points up in a general
way the continuing need

new theory

in advancing numerical simulation.


Third, following on from the trends and challenges in applications software
development, there is the notion that different fields, with their differing software
characteristics, actually require differe
nt types of hardware provision, that a spectrum
of architectures is needed to meet varying application requirements. Clearly there is a
off between the size of hardware system that can be procured for a given budget
and the actual performance that a
given application can achieve on it. This is a
complex, application
dependent argument, but at present informed opinion seems to
be running in favour of providing a range of architectures rather than a single,
inevitably compromised, large system. In parti
cular, at this time of rapid hardware
development, it is likely that some application sectors could immediately make
effective use of particular architectures for which other sectors are not yet ready. This
argues for part of the HEC infrastructure to have

a more experimental character than
in the past. Hitherto, for good reasons, large nationally provided HEC systems have
normally been required to be absolutely robust production systems with demanding
RAS targets, which come with quite a high price tag. It

may be that cost
(and future
proofing) could be optimised by relaxing some of these requirements, at
least when the relevant research communities are prepared to accept an increased
service risk in return for early access to extreme performa

Fourth, we note significant changes in the
modes of usage

of high
end resources. The
traditional batch
oriented focus of these machines is being challenged by the
requirement for real
time, interactive usage. In addition to the
above, we note the potential impact of both c
omputational steering and
visualisation, an impact that is increasingly be driven by "science
pull" rather than by
"technology push".

Finally, we should comment on the complementary relationship between the hig
computing infrastructure central to this submission, and the
emerging computational
infrastructure of the Grid.

The Grid provides an important new resource for
computation, promising a seamless, integrated, and productive environment for
engineers an
d scientists to develop, manage and extract information and knowledge
about their physical systems (e
infrastructure). The Grid should certainly make access
to high
end HEC resources more pervasive; with national HEC services grid
then any plans f
or these services must take account of Grid developments

and vice
versa. The Grid does not, however, provide a vehicle for addressing the key scientific
goals highlighted in this document. These goals demand a high
roadmap that is concerne
d with the development and deployment of the hardware and
software infrastructure needed to enable large
scale computations for breakthrough
science and engineering. Our focus is on simulations that are so memory, CPU, data,
or I/O intensive that they requ
ire dedicated use of a large resource for days to weeks.
end provision should routinely support multi
day runs utilising in excess of
1,000 processors, closely coupled through a high
speed interconnect. Such provision
is critically dependent on the ty
pe of HEC facilities detailed in this document.



Large Facilities Roadmap Criteria

Before outlining the summaries of the scientific case from each of the application
areas central to this submission, we consider a number of more generic areas that
point to

the crucial role of computing infrastructure provision across the entire
spectrum of scientific applications. We focus on the six criteria mentioned in RCUK’s
Large Facilities Roadmap:


The importance of HEC to research in the scientific disciplines;



breadth of science involved across these disciplines;


The position of UK HEC
related activities in an international framework;


The opportunities for training provided through HEC provision;


The contributions to and benefits from UK

industry in HEC, and f


The opportunities for spin
off and exploitation presented through
infrastructure provision.

In each case we assess the impact of future provision by drawing on a number of
examples from the key areas of scientific endeavour outlined in the follow
ing section.

The Importance of HEC

Atomic, Molecular and Optical Physics
Computational research is regarded by the
international AMO community as an equal partner with theory and experiment and
funded accordingly. Information on many (particularly io
nic) species may be obtained
reliably only via detailed first
principle calculations. Large
scale computer codes
have been developed and deployed on the largest HEC systems. These include atom
atom collision codes (cold
collisions), electron
atom and elec
tron molecule collision
codes and a wide range of wave
packet codes for treating high
field and Bose
Einstein Condensates (BEC) problems.

Computational Chemistry:
Improvements in computational resource are expected to
be particularly crucial in the next
five years since many of the methods of
computational chemistry are finding increased applicability in cognate application
areas such as atmospheric science and biology, and these applications present extreme
demands. An additional development, flagged as

a priority for UK chemistry, is the
integration of different (accuracy, length
scale, time
scale) methodologies to enable
unified whole
system approaches to predicting and understanding the properties of
matter, and such activity cannot be contemplated wi
thout an ongoing commitment to
competitive HEC facilities.

Materials Simulation

forms part of the case for support for many HEC procurements
worldwide. It was clearly identified in the Catlow report to SERC in 1992 and is a
major component of the DOE ASCI

and SciDAC programmes [10,11]. It is part of a
recent US
European research initiative [12] that aims to develop the types of research
networks that have been so effective in both the UK materials simulation community
and in European networks in this and r
elated fields

networks in which the UK has
historically always played a disproportionately large role. Indeed, the collective UK
expertise in exploiting HEC to study complex fluids was partly developed through an


EPSRC Network which had substantial indus
trial involvement. The success of this
EPSRC Network subsequently led to the formation of an Institute of Physics Subject
Group in Complex Fluids [13].

Computational Engineering
: Computational engineering requiring high
computing resources remains str
ongly focussed in the general area of Computational
Fluid Dynamics (CFD [14]), although there are encouraging signs of growing interest
from other engineering specialisms. The central issue in fluid dynamics is the
phenomenon of turbulence, which affects t
he overwhelming majority of engineering
flows of industrial interest. Turbulence is detrimental in some applications, causing
additional drag on aircraft and increasing pumping losses in pipework, and is
beneficial in others, leading to improved rates of m
ixing in process plant, increased
heat transfer in cooling systems and increased rates of combustion in engines.
Despite its widespread importance, turbulent flow has proved stubbornly resistant to
mathematical analysis and is notoriously difficult to pre
dict using simple (and even
complex) phenomenological models. Despite recent advances, experimental
techniques are unable to deliver the three
dimensional and time
evolving information
that is required to characterise the flow. As a consequence, the scie
ntific study of
turbulence is based mainly on computational methods, and the greatest advances in
recent years have come from numerical simulations running on the very largest
supercomputers. It is clear that HEC is of central

to the developmen
t of
the subject.

Biomolecular Sciences
: HEC is critical to developing the cutting edge of biological
simulations, at all length scales. Without access to next generation, national facilities
for capability HEC

local/institutional (university or resea
rch institute) facilities
for capacity HEC the subject cannot progress. This would be a major setback in
attempting to use HEC and e
science to integrate the enormous volumes of genomic,
structural and proteomic data that experimental biology is currently
generating. In
particular, there is a need for the parallel development of HEC and e
science to
provide the infrastructure that will enable computational systems biology to become a

Health Sciences & Bioimaging:
This is an area that will benefit
enormously from the
potential integration of HEC and e
science resources, as the large range of length and
timescales to be spanned by such endeavours, and the component complexity of the
systems requires coupling between several levels of description. In
addition to
‘obvious’ target tissues/organs such as the heart, there exist a whole range of systems
(e.g. insulin secreting cells of the pancreas; tumours) which would be open to large
scale simulation, enabling us to develop a systems model which would in
tegrate a
wide range of experimental data and enable us to predict with necessary accuracy the
physiological consequences of possible therapies (e.g. administer a drug that
modulated the dynamics of a given component process, or knock
out/in a gene for a
iven component). Given the highly non
linear behaviour of these complex systems,
such predictions cannot be made without accurate, large
scale simulations.

Radiation Biology
: Radiation oncogenesis is a multistep process including the

stage, the


or expansion processes, the

stage and the
final stage of
. The time scale of these processes ranges from femto
second to many years. Mathematical and computational approaches in radiation


biology have tried to link the init
ial molecular damage with observed lesions in the
form of DNA damage, chromosomal aberrations & mutations, cell transformation,
and relate these to models of cancer induction. These descriptions include simulation
of radiation tracks at a molecular level i
n the environment of the cell, simulation of
the cell nucleus including the full complexity of the DNA macromolecular structures,
the chromosomal domains, descriptions of cell cycle and modulation of gene
expression following DNA damage. These descriptions

require exceptional computer

to determine the spectrum of DNA damage induced by a typical X
irradiation of a mammalian cell requires many billions of coordinates and extensive
CPU requirements. Clearly any attempt at the mechanistic understand
ing of the role of
DNA damage following interaction with the radiation track and the role of
endogenous agents at the molecular level requires the availability of Terascale

Particle Physics
: Lattice Quantum Chromodynamics (QCD), the quantum fie
ld theory
of the strong interaction, is the pre
eminent particle physics application requiring huge
computing capability and world
leading research is impossible without world
HEC resources [15

Environmental Modelling
: The position of the UK en
vironmental modelling research
community in the international arena relies on access to a broad range of research
facilities, of which HEC is key. These research facilities need to be commensurate
with those available to
international colleagues and compet
itors for the UK to retain
its current high international standing. The arrival on the scene of the Earth Simulator
[4] in March 2002, an HEC resource three times more powerful than it nearest rival,
has energised both the US science policy makers and Euro
pe to respond to this HEC
challenge. The UK needs to address this challenge to provide the UK environmental
modelling research community with the HEC infrastructure that will enable it to retain
its international research excellence and to reap the economi
c and social benefit of
that research. Also for climate change policy
making by the UK Government access
to HPC to run global and regional scenarios with credible models is critical.

: Cosmology confronts some of the most fundamental questions in

whole of science. How and when did our universe begin? How does it evolve? How
will it end? These are all questions that have preoccupied mankind since the
beginning of civilisation, but which only relatively recently became accessible to

scientific methodology. Recent advances suggest that significant progress
in answering these and related questions will be achieved in the next few years
through a combination of astronomical observations and numerical simulations.
Section 4 illustrates s
ome of the areas in which, given appropriate computing
resources, the UK is well placed to produce major breakthroughs over the next five
years. None of these key problems can be solved with existing computing resources.
Progress will require an increase
in computational power (including both CPU and
memory) of about one order of magnitude over the next 3
4 years.

Astrophysics concerns the theory, observation and modelling of
astrophysical phenomena. In recent years, the nature of astrophy
sical studies has
been transformed by the advent of massive observational datasets and the ability to
use HEC to model physical phenomena in complex environments. Highlights from the
current UK astrophysical modelling facilities have included the largest e


simulation of star formation, suggesting a new paradigm for formation of close binary
stars, simulations of chemical evolution in neutron star mergers, and studies of the
interaction of jets from active galactic nuclei with surrounding material.


System Science:
In the field of Solar System Science, there is considerable UK
activity in understanding the nature of MHD processes on the Sun, both in the solar
interior (where the magnetic field is generated by dynamo action), in the solar surface
re it emerges as sunspots and intense flux tubes scattered over the whole solar
surface), in the solar atmosphere (where the magnetic field is the dominant source of
energy and is responsible for heating the corona and causing solar flares and coronal


and in the expansion of the atmosphere as the solar wind. The
understanding of the subtle and complex ways in which magnetic fields interact with
plasma is of crucial importance not only for the Sun itself and how it shapes and
defines the Ea
rth’s space weather and climate, but also for our understanding of
similar processes throughout the universe. At present there is a rapid growth in the
use of HEC in this field, increasingly required by the complexity of the MHD and
plasma processes takin
g place in these regimes.

Plasma Turbulence:
end Computing is essential for scientists to gain a first
principles understanding of the heat and particle transport which arises from plasma
turbulence in magnetic confinement fusion devices. The chall
enge of these
calculations is enormous, but improvements in HEC are permitting studies that are
increasingly relevant for experiment. Such calculations will help to optimise
performance in burning plasma experiments which the fusion community is presently

preparing to construct.

Disaster Simulation and Emergency Response
: We are here concerned with disaster
mitigation and emergency response in the most generic sense. Any number of
emergencies could be included in this framework, such as: fires in large bu
ildings and
public facilities; earthquakes in a high population density urban environment;
chemical, biological or radiation type contamination from hazardous spills (or
terrorism); large near shore oil spills in environmentally sensitive locations; major
floods, tidal waves (tsunamis) and so on. To provide an effective response requires
significantly improved scientific understanding of a large number of physical
phenomena (

fires; earthquakes; tsunamis; dispersion of toxins in air, water and
explosions; response of materials and structures to a very large range of
influences). This will inevitably advance scientific understanding and development in
these interdisciplinary areas. This vision is not realisable without HEC.

Breadth of Science

tomic, Molecular and Optical Physics
AMO is directly concerned with ultra
temperature research, the behaviour of matter subject to intense electromagnetic fields
and with ultra
fast measurements. The area provides essential data for a great range of
disciplines including medicine, atmospheric and environmental research, plasma and
fusion research, astrophysics, timing and metrology. The field is also providing
fundamental information related to quantum computing and the quantum behaviour of


aterials Simulation
is an extremely broad field that in techniques and applications
overlaps research programmes in atomic and molecular physics and in computational
chemistry at one extreme and overlaps programmes in engineering and computational
at the other extreme. A wide range of computational approaches is adopted in
materials simulations ranging from atomistic approaches (fully quantum mechanical
to empirical based techniques) to continuum approaches. In the meso
scale regime the
UK has built

up a considerable expertise in soft condensed matter simulations. Studies
embrace the surface structure of materials, chemical reactions taking place on these
surfaces (e.g., a wide range of catalytic processes), and the understanding of many
materials pr
operties (e.g., crack propagation).

Computational Engineering:

of the field comes from the enormous range
of practical applications of CFD, and from the existence of several different
approaches to CFD itself. At the high
end, Direct Numerica
l Simulation (DNS) is
used to solve the governing Navier
Stokes equations using high
accuracy numerical
methods. Inevitably, DNS entails very high computational costs, and both the range of
scales and the complexity of the geometry are severely limited by
the available
computational power, even at the top end of present
day supercomputing. Thus the
approach is not used directly to solve industrial problems, but instead is used as a
scientific tool to obtain detailed information for fundamental studies and

For industrial applications some form of averaging is required in order to reduce the
range of scales that must be represented, and hence to reserve some computational
capacity for handling the geometries of interest. In Large Eddy S
imulation (LES), a
spatial filter is applied at some convenient cut
off scale, and only the large scales of
turbulent motion are simulated in full detail. LES is of greatest value where the flow
is highly unsteady or is dominated by large
scale motion. It

remains computationally
expensive, but is commonly run using clusters of standard Unix workstation or high
end Linux PC hardware.

In Reynolds
Averaged Navier
Stokes (RANS) simulation the small scales of
turbulence are removed using a simple averaging t
echnique. A wide range of well
developed phenomenological turbulence models is available and there is a great deal
of experience in their use and their limitations. The technique is well suited to
complex geometries and has become standard for industrial a
pplications. Many
recent improvements to modelling have been based on data from DNS, and a cascade
of information from DNS to LES to RANS has proved beneficial at all levels.

Biomolecular Sciences:
HEC will involve simulations at a wide range of biologic
length and timescales, including: molecular simulations (nm); sub
cellular and
cellular simulations (μm); and tissue and whole organ(ism) simulations (mm and
above). Thus, biological simulations will be extended beyond their more traditional
domain of s
mall protein molecules to look at more complex, multi
subunit proteins, at
complexes of multiple proteins and larger scale assemblies within cells, and at
modelling of complex tissues such as the whole heart. Applications will include
modelling the normal
behaviour of healthy systems, but also accurate modelling of
disease states (including heart attack simulation and realistic simulation of tumour
growth), and how the dynamics of such disease states may be altered by possible


Health Sciences &
While current work is focussed on single organ
modelling, more complex systems are on the roadmap. Initial simulations of e.g. a
whole human torso have been performed, and it is clear that future studies will need to
integrate simulations, on s
everal length scales, of different organ systems within a
‘virtual human’ This is an area which may be described as computational physiomics.


is already a major area for the use of e
science and GRID computing, as
in the e
Diamond project [18],

a project using the Grid to enable a standardised,
distributed digital mammography resource for improving diagnostic confidence.
However, there are other bioimaging modalities that are likely to make considerable
computational demands in the future. These

include functional magnetic resonance
imaging (FMRI, [19]). Indeed, as has been described in e.g. the work of Chris
Johnson [20] there is considerable work to be done at the interface between
bioelectrical simulation, biomedical imaging and clinical neuro
science that will place
considerable demands on HEC. This leads us towards one of the grand challenges for
biological simulation, namely computational neuroscience and brain simulation

Particle Physics
: Fundamental science, quantum field theory a
nd particle physics
phenomenology, algorithm and program development, the design of leading

Plasma Turbulence:

The scientific tools which are required to understand plasma
transport are widely applicable in many other branches of science:
e.g., astrophysical
plasmas, computational fluid mechanics, and even financial modelling

Disaster Simulation and Emergency Response
: Computational structural and solid
mechanics; modelling of fire and smoke spread in large interconnected spaces
ing turbulent CFD (resolving sufficient length scales for reliable predictions)
including the modelling of combustion and radiation; AI and knowledge based
techniques for command and control operations.

Matching with International Position

Atomic, Mole
cular and Optical Physics
The UK AMO community formerly held a
leading, if not pre
eminent, position within the international AMO community and
was particularly strong in computational AMO physics. In some areas world
research and computer codes c
ontinue to be developed, with key strengths in
Multiphoton and Electron Collision research.

Computational Chemistry
The EPSRC/RSC Whitesides report [21], which is driving
thinking on the directions for UK chemistry research, affirms the historical and
rrent international excellence of the UK community in the core areas of
computational chemistry. Although leading in core theories, the UK community is
noted by the report to be lagging behind in integration of methods for different length
scales for multi
scale modelling, and also in applications in protein folding, reactions
in solution, and biological modelling. These areas, in which the UK must strengthen
its investment to remain competitive, depend partially but critically on the availability
of leadin
g HEC resources.


Materials Simulation
Within the extremely broad field of Materials simulation a wide
range of computational approaches is adopted, ranging from atomistic to continuum
approaches. In the meso
scale range, the UK has built up a considerabl
e expertise in
soft condensed matter simulations. Here the UK community has been particularly
effective at exploiting HEC for materials simulations and has a strong reputation for
developing new computational techniques and algorithms. The recent Internati
Review of UK Research in Materials [22] commented on the UK’s strength in
atomistic and meso
scale modelling, maintained despite intense international

Computational Engineering:

standing of UK research in CFD
remains v
ery high. Since the earliest days, UK researchers have been at the forefront
of developments in CFD and especially in RANS
level turbulence and combustion
modelling. UK researchers continue to play a leading role in DNS for reacting and
reacting flow
s, having recently carried out some of the largest simulations in the
world, and are making significant contributions to the development of LES as an
industrial tool.

Biomolecular Sciences
: In biomolecular and whole organ simulations the UK has a

of research groups which are world leaders. However, there is a real danger
that we could be overtaken by competitors in both the USA [23] and Japan who will
have access to facilities considerably in excess of those provide by HPC
. For
example, recent si
mulations of the visual protein rhodopsin at Sandia National
Laboratories [24] in the USA exceed those possible within the UK by a factor of ca.
5x. In the absence of an enhanced high
end facility this situation will become
commonplace and the UK’s positio
n in biological simulations will become marginal.

Health Sciences & Bioimaging
: The UK has a world
leading position in whole heart
simulations, with major computational groups in Leeds and Oxford (the latter in
collaboration with Peter Hunter’s group in N
ew Zealand). There is considerable
potential for building upon this expertise, thus further developing UK strengths in
computational systems biology.

Radiation Biology
: Radiation damage to DNA and the repair thereof is being
investigated at LBNL by the De
partment of Radiation Biology and DNA Repair in the
Life Sciences Division [25]. Large simulations of radiation damage to DNA have
been performed, allowing theoretical models to be tested and/or verified, and insights
into the structure of DNA at the level

of the chromatin

Particle Physics
: The UKQCD collaboration is a leading international player,
producing as many of the top
cited papers on the “hep
lat” archive as any other
collaboration, and more than most. Collaborators are regularly invited a
s plenary
speakers at the major international particle physics conferences.

Environmental Modelling
: The UK’s pre
eminence is well recognised in
environmental modelling fields from climate modelling to the study of the physical
behaviour of rocks and mine
rals in the Earth’s interior. The UK’s environmental
modelling community has an impressive publication record and makes extensive and
critical contributions to international research programs such as IPCC.


: Recent reviews of Physics research in
the UK have repeatedly singled out
theoretical astrophysics, cosmology in particular, as an area of internationally
recognized excellence. Most HEC
based cosmology work in the UK takes place
within one of two consortia

Virgo and COSMOS. Virgo is a colla
boration of
researchers in five countries which is based in and managed from the UK [26]. The
consortium was created as one of the original
Grand Challenge

projects in EPSRC's
High Performance Computing Initiative in 1996. Since then, it has emerged as the

world leader in the field of N
body simulations of dark matter distribution. The
impact of the consortium in world science is illustrated, for example, by the fact that
two of the five most cited UK scientists in the entire Physical Sciences during the pa
fifteen years (Efstathiou and Frenk) are members of the consortium. Currently,
Virgo's computing resources include the
Cosmology Machine
, a dedicated
supercomputer sited in Durham as well as access to the Max Planck IBM Regatta
system in Garching. The
other consortium, COSMOS, is an entirely UK effort.

Solar System Science:
The UK Solar System modelling community has an
exceptionally strong international reputation in magnetohydrodynamics,
helioseismology and magnetosphere
atmosphere interactions. Fu
rthermore, this is
very much an international community, where UK scientists have close collaborations
with colleagues abroad and expect to play key roles in a series of future European,
Japanese and American space missions.

Plasma Turbulence:

activity in the understanding of plasma turbulence
through exploitation of HEC has been active in Europe and especially vigorous in the
US where a number of first principles approaches have been funded under the US
DOE SciDAC initiative. Traditional UK str
engths have been in analytical theory, but
there have been recent moves to exploit HEC in the UK through collaborations.
Access to improving HEC is extremely vital for this activity.

Disaster Simulation and Emergency Response:
UK researchers in the area o
structures in fire are recognised leaders in the world with University of Edinburgh fire
research group internationally known for the fundamental developments in fire
science and fire safety engineering.

Opportunities for Training and Outreach


Molecular and Optical Physics
The research groups involved in the AMO
consortia using HEC facilities include both research students and post
fellows. It is important that these younger researchers are trained to use HEC systems
and to develop l
scale scientific programs using up
date techniques. Interaction
with HEC support staff encourages the dissemination of new methods and concepts
derived from computational science research. This training is invaluable whether they
continue with scie
ntific careers or leave for industry.

Computational Chemistry
A particular feature is the Collaborative Computational
Project structure, which supports the process of deploying new theories and
algorithms in software that can be used by the community. CC
P1 (electronic
structure) [27], CCP5 (simulation of condensed phases) [28] and CCP6 (molecular
quantum dynamics) [29] all promote UK software, and underlying activity in theory
and methodology, that is recognized to be world
leading. These communities are



exploiting existing supercomputer facilities to great effect; for example, the
ChemReact HPC
consortium [30] integrates the methods promoted by CCP1 and
CCP6 to tackle grand
challenge problems in chemical dynamics. A key activity of
the CCPs is the

training of postgraduate students and postdoctoral researchers in the
methods of high
performance computing.

Computational Engineering:
UK academic groups continue to provide advanced

for a large number of CFD developers, users and applications
specialists for
UK research and industry.

Biomolecular Sciences
: The potential user base in the UK is ca. 20 research groups.
However, of these possibly only 5 or so groups are currently signed up to use HPC
In part, this is an issue of people having ‘t
rimmed’ their aspirations to match the
facilities readily available. There is a considerable opportunity for training a new
generation of HEC

and GRID
enabled postdocs and graduate students in
computational biology. These people will be able to use (and w
ill expect to use) large
scale facilities comparable to those in the USA and Japan and thus will deliver world
class science.

Particle Physics:
Many graduate students and PDRA's go on to work in other
scientific areas and industry.

Environmental Modellin
: Research groups involved in environmental modelling
have a strong record of training via their PhD programmes. The UK centres of
research excellence provide their PhD students and PDRA’s with a rich experience of
HEC and sophisticated environmental mode
lling. The employability of graduates and
postgraduates in the field of environmental modelling is high as shown from records
(from Universities and the research councils) of future employment which features
both industry and government agencies (Met Offic
e, Environment Agency).

: Cosmological simulations also play an important role in scientific
outreach. Astrophysics and cosmology stimulate the public imagination more than any
other physical science because of the remoteness of the frontiers the
y explore, because
of the beauty and immediacy of astronomical images, and because of the depth of the
questions they address. Supercomputer simulations convey our current picture of
cosmic creation and evolution both accurately and vividly; they are routi
nely used by
museums, planetaria and popular science magazines to stimulate public interest in and
awareness of science. Cosmological supercomputing is a core activity at many of the
largest scientific supercomputer centres in Europe and North America. Sc
trained in this field find their skills in great demand in most areas of technical and
industrial supercomputing.

Plasma Turbulence:

A number of PhD studentships are available in the UK, bringing
young scientists into this challenging area and pr
oviding sound training for possible
future careers either within fusion research or in wider industry.

Disaster Simulation and Emergency Response:
Many graduates and postgraduates
trained in fire safety engineering, CFD modelling of fires and computationa
modelling of structures in fire have gone on to work in industry and academia


Contributions to and Benefits from Industry

Atomic, Molecular and Optical Physics
The lighting industries in the US and
Australia have shown strong interest in
ongoing work to improve the accuracy of
atom collision calculations. There is also interest from UK industry carrying
out plasma etching in work on electron
molecule collisions. Although the latter
interest is focussed on experiment these areas wi
ll all benefit from theoretical work
facilitated by increased HEC provision.

Computational Chemistry
: The rational drug design problem in particular represents a
link to the pharmaceutical industry that will increase in importance. Already the UK
y is well positioned through collaborations, and through the distribution of
key software to industry. At present, most industrial HEC is done with commodity
clusters; however, software for a future generation of commodity hardware will have
to be develope
d and tuned today on an advanced machine.

Computational Engineering:
There is a very strong link with

through CFD
application areas such as external aerodynamics of aircraft and motor vehicles
including Formula 1 racing cars, internal aerodynamic
s of jet engines and other gas
turbines as well as automotive engines, hydrodynamics in marine engineering, and
structure interaction for the offshore oil industry. The link is two
way: industry
requires robust, efficient and accurate CFD tools, and

in return industry provides
academic CFD developers with new and challenging problems.

of CFD
research has come mainly through the development and licensing of CFD codes, and
through companies offering advanced consultancy services. Commercia
l CFD codes
are increasing in reliability and ease of use, and have enabled the introduction of CFD
to a large number of industries. CFD applications are becoming increasingly complex
and include mixing and reaction in process plant for the chemical and b
industries, as well as turbulent combustion, where the applications include gas turbine
combustors for aero, industrial and marine engines, internal combustion engines and
the prevention of accidental explosions in the petrochemical industry.

Biomolecular Sciences
: The main benefits will be to the pharmaceutical industry.
These will be twofold. Firstly, biological simulations in the academic sector will
provide proof of principle and development of methodologies. Actual industrial

are likely to be more capacity
oriented and will be likely to take place in
house. The second benefit will be a supply of suitably trained postdoctoral researchers
to undertake high
end simulation application studies within the pharmaceutical sector

ticle Physics
: Collaboration with IBM in the design of QCDOC, with IBM
looking to exploit these ideas in the design and implementation of the BlueGene/X

Environmental Modelling
: The link between the environmental modelling community
and the busine
ss community is strong especially via the strong collaboration with the
Met. Office, and other governmental agencies like DEFRA. The economic benefit to
society of quantifying forecasts, on whatever timescale, is enormous. By providing
probabilistic result
s for impact models or agencies involved in assessing impacts of


extreme events or climate change adaptation or mitigation strategies can be

Plasma Turbulence:

Besides the direct benefits to a possible fusion power industry in
the future, the

students who emerge with PhDs in first principles modelling of
turbulence in fusion plasmas have developed excellent mathematical and modelling
skills which are highly valued in wider industry.

Disaster Simulation and Emergency Response
: Strong links ex
ist with Ove Arup &
Partners (consulting engineers), software vendors (ABAQUS, ANSYS, CFX, Fluent
etc.) and many other international organisations, such as NIST (USA), TNO (NL),
IRSN(France). All are very interested in collaborating to further develop the
necessary technologies for disaster simulation and response. Government bodies such
as ODPM and HSE and emergency services interested and involved as beneficiaries
and users.

Opportunities for Spin off and Exploitation

We should perhaps stress that given

HEC cuts across

disciplines, it is difficult to
provide a simple, coherent account of its exploitation potential. Here however are
some selected opportunities:

Atomic, Molecular and Optical Physics
Computational research in the AMO area
using HEC
has a large impact in the planning and interpretation of experiments
carried out at UK large
scale facilities such as synchrotrons (Daresbury SRS,
Diamond) and at the Central Laser Facility. The proposed development of UV FEL
facilities envisioned in the 4
GLS project will benefit particularly from theoretical
work based on Floquet methods. These approaches are well suited to the longer pulse
lengths and higher frequencies of an FEL. Theoretical work on resonant harmonic
generation has already stimulated in
terest within the experimental community at the
possibility of observing the predicted enhancements.

Electron scattering calculations needed to provide fundamental data for astrophysical
studies are already underway. Industrial applications, for example i
n the lighting
industry, will also follow from increased accuracy for treating fundamental collision
processes. A range of applications is given in the references listed in the appendix.

Computational Engineering:
For the future, CFD will continue to plac
e heavy
demands on computer power at all levels. DNS will require access to the most
powerful HEC machines available in order to contribute to understanding at the
highest level. LES and RANS will be applied to more difficult and more complex
problems using ever more powerful desktop hardware. There will be a
growing demand for better and more accurate solution algorithms and physical
submodels, and there will be greater integration with other disciplines, most notably
in materials science and

in engineering design. This will lead to even greater
opportunities for exploitation.

Biomolecular Sciences:
The most likely spin
off is the development and possible
commercialisation of an integrated framework for multi
scale biological simulation in

enabled HEC system (see, for examples, [31]).


Particle Physics
: While lattice QCD is primarily of pure science interest, we note the
exploitation by IBM of what it has learned from QCDOC involvement.

Plasma Turbulence:

Plasma turbulence calculatio
ns will play an invaluable role in
guiding physicists and engineers to optimise the design of fusion power plants.

Disaster Simulation and Emergency Response:
Major spin
offs in development of
fundamental understanding of fire behaviour structural respons
e to fire, design of
structures for fire and earthquake resistance (smart structures) and integration of
sensors and computations.



The Scientific Case: Selected Applications

Atomic, Molecular and Optical Physics

The present frontiers of AMO science [32
] are characterized by extreme conditions:
cold temperatures, ultra
intense light and ultra
fast light pulses. In each of these
areas the interactions between atoms, molecules and light is studied in a highly
nonlinear regime leading to novel effects

and new insight into the fundamental
properties of matter. Two of the major areas of theoretical AMO physics that are
developing rapidly in response to innovative and ingenious experimental observations


temperatures physics (Bose and Fermi conde
nsates, atom and ion traps,
atom optics, cold

The properties of a Bose
Einstein condensate formed
by cooling magnetically
trapped alkali metal atoms are the focus of intensive
theoretical and experimental studies, with theoretical models of th
e condensate
providing an essential guide for the interpretation of the experimental
observations. The trapping of a BEC in an optical lattice opens a wide range of
new possibilities including quantum computing devices. This is also an excellent
for studies of coherent matter transport in periodic condensates. The
challenge of following and visualizing the temporal evolution for more general
BECs, including those contained within an optical lattice requires enhanced
Terascale computing facilities.


High field multiphoton physics (matter subject to intense ultrashort laser pulses

femtosecond and attosecond physics).

probe laser spectroscopy has allowed
the motion of molecules to be observed in real time with a resolution of a few

(1fs = 10
secs). This femtosecond chemistry has stimulated new
insight into chemical processes and the motion of molecules in terms of wave
packets. In the past year, the goal of studying the motion of electrons bound in
inner orbits of atoms has been
realized by the creation of laser pulses with a pulse
length of a few hundred attoseconds e.g. Attosecond XUV pulses have been used
to ionize Krypton atoms. Theoretical techniques for studying systems subject to
intense electromagnetic fields with more tha
n two active electrons are limited to
Floquet or non
Hermitian Floquet dynamics approximations. Both techniques
approximate the time
dependence and are valid only for laser pulses longer than
10 fs. Wave packet calculations have been used with great succes
s to study
systems with up to two active electrons. There is a huge computational and
theoretical challenge in developing more general methods for solving the time
dependent Schrödinger equation for systems like Krypton, particularly when
taking into accou
nt relativistic effects. Approaches such as time
dependent R
matrix theory are being developed and will certainly require the greatest
computational power available.

The generation of antihydrogen atoms is also stimulating interest in the formation and
roperties of antimatter. More traditional topics, such as electron collisions with
atoms, molecules, clusters and surfaces are experiencing a renaissance as a result of
the development of FEL lasers and new particle beams. Computational AMO science
is well
placed to take advantage of the increased power of HEC systems. The


interactions are known, the many
body problems involve a limited, albeit large,
number of interacting particles. A wide range of important fundamental and practical
problems is likely to

be tractable using the current or next generation of computers.

Computational Chemistry

The methods of computational chemistry continue to be driven by both advances in
theory and the cycle of availability of increasingly capable computers. Both of the
are essential for the continuation of the leading position that the UK community
enjoys. The principal current domain of HEC in chemistry is the determination and
interpretation of the properties of molecular matter using models that are, or are based
n, first
principles physics. Some of the problems in solid
state and surface chemistry
demand a treatment that embraces the periodic or bulk nature of the material; here we
address science that is concerned with the behaviour of individual molecules, in t
gas phase or solvated. We note the increasing convergence of methodology between
the solid
state and gas
phase disciplines. Other problems are explicitly biological in
nature, such as protein structure and enzyme catalysis. Increasingly, these are
ming accessible through the methodology of computational chemistry through
algorithmic developments and improved computers.

A particular feature is the Collaborative Computational Project structure, which
supports the deployment of new theories and algori
thms in software that can be used
by the community. CCP1 (electronic structure), CCP5 (simulation of condensed
phases) and CCP6 (molecular quantum dynamics)

all promote UK software, and
underlying activity in theory and methodology. These communities are a
lso exploiting
existing supercomputer facilities to great effect e.g. the ChemReact HPC

integrates the methods promoted by CCP1 and CCP6 to tackle grand
problems in chemical dynamics.

Areas identified for development include the in
tegration of methods for different
length scales for multiscale modelling, plus applications in protein folding, reactions
in solution, and biological modelling. These areas depend partially but critically on
the availability of leading HEC resources. Th
ere is great potential in each of these
interdisciplinary areas, provided that the appropriate high
end computing resources
become available: already in place
is the expertise

in core methods, together with
some cross
application and cross
platform integra
tion delivered by e
middleware and pilot projects.

An important recent area of advance has been in improved scaling of computational
resource demand with system size. Linear
scaling density functional theory can now
be applied to 1000
atom system
s within molecular dynamics simulations. More
recently, advances in many body methods such as MP2 and CCSD(T) mean that these
more accurate methods can now also be applied to systems with several hundred
atoms; an advanced computational resource will allo
w deployment on significant
problems of dynamics and structure. A complementary methodology, QM/MM,
presents a hierarchical multiscale solution to problems where the chemistry is
localized at an active site; an important example is enzyme catalysis. Improv
computational resources will allow improvements in accuracy through more
sophisticated QM models and more realistically sized QM regions. There will also be


a need for significantly increased computational capacity, for the application of such
ns to rational drug design where large numbers of target molecules need to
be explored.

Explicit consideration of the quantum dynamics of chemical reactions is essential for
the first
principles computation of kinetic data such a thermal rate constants.

For the
many small
molecule reactions that take place in the atmosphere, determination of
these rate constants is an essential input to the modelling of large
scale processes in
the atmosphere, and experimental measurement of all reactions at all temperat
ures is
not practical. Such modelling is in turn of key importance in understanding and
predicting the future behaviour of the atmosphere. Key advances are being made by
the UK community in solving the quantum dynamics problem to high accuracy,
building o
n earlier successes in treating simple triatomic reactions; projected work
will demand a machine with larger memory and significantly increased speed.

Simulation of condensed phase problems continues to be ubiquitous and successful.
An important emergi
ng advance is the extension to different length scales. QM/MM
represents the microscopic end of this process; in complement, CCP5 is developing a
mesoscale simulation code that will integrate atomistic and hydrodynamic treatments.
Full exploitation of thi
s and other advances in force
field classical simulations will
demand continually improving resources.

Materials Simulations

Covering the extremely broad field of Materials simulations has resulted in the
adoption of a wide range of computational approac
hes ranging from the atomistic to
the continuum. The capability of all these techniques has advanced enormously in the
last decade

we can study every surface structure of every material using QM
techniques plus chemical reactions taking place on these su
rfaces. Simulations using
empirical potentials have advanced our understanding of many materials properties

we now understand why brittle cracks typically propagate at speeds substantially
lower than the sound velocity. Despite these successes, the enorm
ous scope of
materials modelling means that there are still countless challenges to be confronted.
Examples include:


Most crack simulations are still only 2
dimensional. With increased computer
power these simulations can become fully 3
dimensional allow
ing issues such as
the structure of steps on the crack edge and their effect on a range of materials
properties such as resistance to chemical attack to be addressed.


Accuracy will continue to be a critical issue. Many problems require QM
accuracy, with
the challenge of moving to larger systems more acute since the
computation scales as the cube of the number of atoms in the system using
conventional DFT approaches. These approaches will be replaced with linear
scaling techniques; the UK is well positione
d to spearhead this transition with
linear scaling codes under development [33


Although DFT allows sophisticated QM modelling of large systems, it is restricted
to ground
state electronic configurations. This provides a severe limitation on our
ty to model problems such as radiation damage and excited state contributions


to surface processes such as catalysis. There is a pressing need for a computational
scheme for excited electronic states to allow these problems to be studied.


A billion atoms
represents only a 100 nanometre cube of material

so far from
relevant lengthscales in complex fluids that alternative approaches are required. A
range of simulation techniques provide a hierarchical approach to studying
complex fluids, yet such approache
s break down when there is a complex
interplay between phenomena at the different lengthscales. While hybrid QM/MM
modelling schemes provide a mechanism for dealing with these problems, the
matching between the QM and MM regions is extremely difficult in m
simulations. Grid technologies will have a significant impact in speeding the
development of hybrid modelling schemes by providing transparent access to
compute resources, rapid communication between different components of the
simulation and acce
ss to data from previous simulations allowing expert systems
to be developed.

Over the last decade, research has focussed primarily on larger simulations. While
there is a continuing drive to allow even larger system sizes to be studied, such
systems pre
sent a further problem

that of complexity. There are a larger number of
possible atomic configurations for the system to explore which results in longer
timescales for physical and chemical processes. The challenge of materials modelling
is not just to m
ove to larger and more complicated systems but also to perform much
longer simulations and, more importantly, to develop new techniques for searching
the increasingly complex configuration spaces of these systems.


Nanoscience (defined by the
ability to control matter on sub
100nm lengthscales with
molecule precision) is a rapidly developing and highly active field worldwide,
across several traditional disciplines in the physical and biological sciences and
engineering. This activity al
ready includes a significant computational effort
alongside the experimental work, and the computational component is expected to
grow significantly over the next few years. Theory and simulation are essential
components of understanding new science and de
signing new technologies in a
situation where materials behave in radically new ways, and many important
quantities are still inaccessible to direct experimental measurement.

The reason for the importance of the size range from atomic dimensions up to 10
is that it encompasses many outstanding scientific and technological challenges across
this range of disciplines, including

The development of new, smaller and faster, information
processing devices that
will be needed to extend the four decades of ex
ponential reduction (in size and
price) since the development of the transistor and the integrated circuit (“Moore’s

The mechanistic understanding of the part that biological macromolecules and
assemblies of macromolecules play in the machinery of a

living cell

The combination of these two fields to sense and monitor biological processes,
and inform medical diagnosis, on an unprecedented fine scale.


However, some of the same factors that make nanoscience so important also make it
ecially challenging from a theoretical and computational point of view. First, the
nanostructures involved are very large by comparison with conventional molecules,
but very small compared to bulk materials; therefore many of the usual
approximations and
computational techniques that are used for small molecules or
bulk materials are not applicable. Second, at sub
100nm lengthscales the properties
of matter are intimately coupled to the atomic structure, which not only varies
between instances of nominall
y identical nanostructures but also fluctuates strongly in
time. Third, the dynamics of nanostructures may cover an extremely wide range of
timescales, from below 1ps (10
s) in response to electronic transitions or changes in
charge state, to seconds or
more when considering the growth or organization of
assemblies of nanostructures. Finally, the functioning of a nanoscale experiment or
device may involve different parts with radically contrasting properties (e.g. an
interface between a “hard” and a “so
ft” material).

There are several types of nanostructure whose properties are of interest as the
building blocks of this new science, and of the associated emerging technologies

Quantum dots and nanocrystals,
Quantum wires and nanoscale logic devices,
noscale electromechanical devices, and Biological molecules, such as DNA or
proteins. Such building blocks are increasingly being assembled into complex
combinations; for example, nanoparticles coated with organic molecules are being
proposed as bio
tible markers, in photo
voltaic devices, or as electronic
components. It is apparent that a range of techniques from different simulation fields
is required to analyse this kind of system. The scope of calculations that will be
needed is developed in the
detailed applications evidence; the ability to model both
excitations and atomic structure, and to integrate models at different time

and length
scales, will be key components of success.

Computational Engineering: Fluid Dynamics

Computational engineeri
ng requiring high
end computing resources remains strongly
focussed in the general area of Computational Fluid Dynamics (CFD), where the
central issue is the phenomenon of turbulence. Despite its widespread industrial
importance, turbulent flow is notorio
usly difficult to predict. The scientific study of
turbulence is based mainly on computational methods, and the greatest advances in
recent years have come from numerical simulations running on the very largest
supercomputers. CFD methods include Direct

Numerical Simulation (DNS) for basic
research, Large Eddy Simulation (LES) for advanced industrial simulation, and
Averaged Navier Stokes which has become the standard for industry. CFD
is used routinely in areas such as external aerodynamics of

aircraft and motor vehicles
including Formula 1 racing cars, internal aerodynamics of jet engines and other gas
turbines as well as automotive engines, hydrodynamics in marine engineering, and
structure interaction for the offshore oil industry. Si
nce the earliest days, UK
researchers have been at the forefront of developments in CFD and especially in
level turbulence and combustion modelling. UK researchers continue to play
a leading role in DNS for reacting and non
reacting flows, having rec
ently carried out
some of the largest simulations in the world, and are making significant contributions
to the development of LES as an industrial tool. UK academic groups continue to
provide advanced training for a large number of CFD developers, users


applications specialists for UK research and industry. For the future, it is clear that
CFD at all levels will continue to make major demands on computing resources as the
industrial problems become ever more complex.

Biomolecular Sciences

The UK bi
omolecular simulation community is in a good position to respond to the
challenges posed by post
genomic structural biology. In particular we wish to exploit
the main strengths of our discipline, revealing emergent physicochemical details
implicit in stati
c structures. Future challenges include: whole cell simulations; protein
folding; protein/drug and protein/protein interactions; enzyme dynamics and
reactivity. This is an exciting area of research, with great potential to expand in the
context of ongoing
HEC and e
science. New (better scaling) methodologies do need
to be developed, to enable us to explore a wider range of timescales and to address
more complex macromolecular and cellular assemblies. This area has diverse
computational needs, including

computing for high
end projects, but also
high performance

computing (ideally GRID
enabled) as an essential
component of an integrated post
genomics approach to simulation.

Computational Systems Biology

HEC will play an increasingly impor
tant role in the area of Computational Systems
Biology (also known as integrative biology). The overall aim of this area is to use
computational approaches to assist in the development of predictive biology based on
the large volumes of data emerging from
genomic and post
genomic approaches to
biology. In particular, CSB will require large scale computation to attempt in silico
modelling of whole cells. This area of research is of major importance within the
context of the 10
year plan of BBSRC and of FP6 o
f the EC.

Health Sciences & Bioimaging

In addition to simulations at the molecular, and multi
molecular levels, it is now
feasible to perform meaningful
simulations of physiological properties at the
cellular level and beyond
. For example, models of the
electrical and mechanical
properties of an entire heart have been performed. The next few years will see such
models extended by inclusion of molecular biological information on expression
levels of individual membrane protein species in different regions
of the heart. Such
models will enable reconstruction of the molecular and cellular basis of disease states
e.g. ischaemia, congestive heart failure. Thus, simulation results will provide a major
tool in the search for better treatment strategies for patien
ts with heart disease. Similar
approaches may be taken to other organ systems, e.g. pancreatic β
cells and diabetes.
The more general importance of such simulations is that they enable us to bridge the
gap between genomic and structural studies of proteins

and integrative physiological
studies of cells and whole organisms. Thus they are an essential component of any
rigorous approach to molecular medicine.

Radiation Biology

A major concern in radiation biology is the understanding and quantification of
rmful effects of ionizing radiation on earth and the earth atmosphere from natural


and man made sources and its considerable impact on high
speed civil transport and
space exploration from galactic cosmic rays. While radiation biology over the past
ntury has made great advances in providing a detailed description of radiation
action in the cell, the progress has been slow, uncertain, intuitive and largely
descriptive. In order to achieve real insights into and understand the controls and
processes in
volved in biological phenomena, mathematical and computational
methods offer the precise tool needed to design appropriate experiments. In the recent
past there has been a rapid surge in mathematical theories and biophysical models
along with the advances
in laboratory and clinical cancer research to provide
predictive descriptions.

Ionizing radiation is both useful and a harmful agent. It kills cells down to very low
doses, and at low doses can cause damage leading to cancer induction; it is an

agent for treatment of cancer and could also be used as a radioprobing tool
for design of novel drugs in the treatment of cancer. Over the past decade with the
accelerated progress in molecular biology techniques and advances in theoretical and
onal methods attention has become more focused on mechanistic studies
and interpretation of effects of ionising radiation. To this end, theoretical and
computational methods have provided the tool to investigate those parameters of
ionising radiation that
predominantly determine the nature and magnitude of the final

Radiation oncogenesis is a multistep process including the

stage, the

or expansion processes, the

stage and the final stage of
. The time scal
e of these processes ranges from femto second to many
years. Mathematical and computational approaches in radiation biology have tried to
link the initial molecular damage with observed lesions in the form of DNA damage,
chromosomal aberrations & mutation
s, cell transformation, and relate these to models
of cancer induction. These descriptions include simulation of radiation tracks at a
molecular level in the environment of the cell, simulation of the cell nucleus including
the full complexity of the DNA
macromolecular structures, the chromosomal
domains, descriptions of cell cycle and modulation of gene expression following
DNA damage. These descriptions require considerable computer resources e.g. to
determine the spectrum of DNA damage induced by a t
ypical X
irradiation of a
mammalian cell requires many billions of coordinates and extensive CPU
requirements. Such data then could become the input to ascertain the fate of the cell
and subsequent biological effect. Clearly any attempt at the mechanisti
understanding of the role of DNA damage following interaction with the radiation
track and the role of endogenous agents at the molecular level requires the availability
of Terascale computers.

Particle Physics: Quarks, Gluons and Matter

Quarks and gl
uons are permanently confined by the strong interaction into bound
states called hadrons (for example, protons and pions). In consequence, parameters of
the Standard Model of particle physics, quark masses and the strengths of the decays
of one quark into
another, cannot be measured directly, but must be deduced from
experiment with the input of theoretical calculations incorporating nonperturbative
strong interaction effects. Such calculations are only possible using numerical (lattice)
simulations of Quan
tum Chromodynamics (QCD), the quantum field theory of the


strong interaction. Lattice QCD is the pre
eminent particle physics application
requiring huge computing capability and world
leading research is impossible without
class HEC resources [15

UK scientists formed the UKQCD Collaboration for lattice simulations at the
beginning of the 1990s and have established UKQCD as a major international player
[36]. The collaboration is currently working with Columbia University, USA, on the
n a chip) project, with strong involvement from IBM, which will
give UKQCD a 5

resource from June 2004.

Scientific opportunities to be addressed by this and future machines include:

Structure and interactions of hadrons: reproducing the observed
masses of
hadrons, including states whose quantum numbers demand the existence of
gluons; calculating distributions of quarks and gluons in hadrons.

Precision testing of the Standard Model. Nonperturbative lattice QCD calculations
(for example of neutral
meson and K
meson mixing) are vital to isolate and
extract underlying Standard Model parameters. Terascale computing will provide
a substantial increase in the precision of such calculations and also enable more
demanding calculations, such as those need
ed to understand the so

rule and the Standard Model description of CP violation.

Matter under extreme conditions. A new state of matter, the quark
gluon plasma,
is expected at high temperature and/or baryon density. Lattice QCD can be applie
to understanding this new state and its influence on experimental results. Progress
here requires both Terascale computing power and new algorithmic ideas.

UKQCD already has a working QCDGrid for storing and distributing its data [37] and
is a leading me
mber of the International Lattice Data Grid (ILDG, [38]).

Environmental Modelling

The future HPC requirements for environmental modelling research have been
described in many different fora, from the worldwide response to the challenge of
Japan’s Earth S
imulator to UK government committee on climate research [39
The following examples illustrate the essential drivers of the UK academic
community’s HPC requirement for environmental modelling:

The need for very high resolution models for understandin
g and quantifying
prediction of extreme events and for the regional assessment of the impact on
society and economy of climate change:
The forecasting of extreme weather
events (storms, flooding etc) is of major economic value for the UK. A critical
ement for advancing the forecasting of such events is not only improving the
science underlying the forecasting on
short timescales

(hours or days) but also to
improve models of changes to the patterns of occurrence on
longer timescales

(seasonal, decadal
or century). The former needs improved models and methods
for the assimilation of data (radar, satellite); such models, with resolutions of the
order of a few kilometres, are intensely computationally demanding. On
, improved regional clim
ate models, embedded in global climate
models, need to be developed and the effect of global warming on high


weather events better understood. UK research collaborations, such as UWERN
(University Weather Research Network), with its strong well inte
grated links to
the UK Met Office, are in the forefront of scientific progress that underpins the
development of next generation local and regional models for forecasting extreme

The need for current climate models to move towards Earth System Mod

atmosphere and ocean resolutions associated with current state
climate models do not adequately represent key aspects of the climate system (e.g.
the influence of ocean eddies, El
Nino or tropical cyclones). High resolution
phere and ocean models have already demonstrated significant
improvements in the representation of processes such as the North Atlantic storm
track and the definition of the Gulf stream. These high resolution components now
need to be coupled to other comp
onent models representing processes in the Earth
System, hitherto very poorly represented, such as chemistry or land surface or sea
ice. Doubling the spatial resolution of models leads to a 10X increase in
computational requirement while coupling the multi
ple different high resolution
Earth System modelling components will demand a 100X increase in
computational requirement. High performance computers for future Earth System
modelling experiments will be key to the evaluation of climate change, its impacts
and the development of strategies to adapt to, or mitigate those impacts. The
collaboration of the
UK academic community,

for example NCAS (NERC Centres
for Atmospheric Science), with the
Met Office’s
leading Hadley Centre as
well as with Europe, US
A and Japan, means that the UK is at the forefront of the
use and development of the next generation Earth System models.

need for quantifying uncertainty
; Environmental models are inherently
imperfect owing to physical processes that are either not c
ompletely understood or
yet to be adequately represented because of limited computer power. The
consensus approach to solving this and related problems is to assume that the
uncertainty can be estimated by averaging an ensemble of model experiments.
This e
based approach is used for a broad range of environmental models
and experiments and on a broad range of time scales, from days to centuries.
Ensemble experiments are computationally expensive but there could be
enormous economic benefit as well as

an advance in our understanding from
improving the reliability of models or estimating uncertainties in forecasts.