The Grid
Meeting the LHC
computing challenge
Gavin McCance
University of Glasgow
RSE
6
th
February
2002
RSE 6 February 2002
Gavin McCance
1
/
23
Outline
Scale of the LHC computing challenge
Grid ‘Middleware’
Data Replication
Experimental testbed
RSE 6 February 2002
Gavin McCance
2
/
23
LHC computing
challenge
Typical experiment:
2 MB per event
2.7x10
9
event sample
5.4 PB/year
Up to 9 PB/year Monte Carlo samples
Very large storage and computational
requirements
CERN can handle max of 1/3 of this!
RSE
6
February
2002
Gavin McCance
3
/
23
…computing challenge
Distribute data store and compute
resources
Take advantage of existing local clusters
and local infrastructure
Easier to get funding for local clusters,
particularly cross
-
experiment or cross
-
disciplinary compute resources
RSE
6
February
2002
Gavin McCance
4
/23
Tiered model
Tier
2
Centre
CERN Computer
French Regional
Centre
Italian Regional
Centre
Institute
Institute
Institute
Institute
Tier
2
Centre
Tier2 Centre
Tier
2
Centre
Tier
1
Tier 3
ScotGRID
Tier
2
US Regional Centre
Tier
0
RAL Regional
Centre
Basic reconstructed data
Higher level analysis data
and Monte Carlo
Tag data and Monte Carlo
RSE 6 February 2002
Gavin McCance
5
/
23
UK Grid
Tier2 Centre
Institute
Institute
Institute
Institute
Tier
2
Centre
Tier2 Centre
Tier2 Centre
Tier
3
ScotGRID
Tier
2
RAL Regional
Centre
Higher level analysis data
and Monte Carlo
Tag data and Monte Carlo
Tier 1
Basic reconstructed data
GridPP collaboration
RSE
6
February
2002
Gavin McCance
6
/23
GridPP
RSE
6
February
2002
Gavin McCance
7
/
23
…GridPP
£
17
M three year project
Working in collaboration with the
EU DataGrid project
Middleware production
Integration of middleware technologies
into HEP experiments
Validation of Grid software
RSE
6
February
2002
Gavin McCance
8
/23
Middleware
What is middleware…???
Application programs
–
gridopen() call
Data access specifics
–
HPSS, Castor
Job submission specifics
–
PBS, LSF
Specific security procedures
Grid middleware
Layered API’s.
Transparent security.
Transparent data access.
Intelligent use of
distributed resources.
RSE 6 February 2002
Gavin McCance
9
/
23
Middleware Activities
GridPP ~mirrors EU DataGrid:
Workload Management
What jobs go where?
Data Management (*)
Where’s the (best) data?
Information Services
What’s the state of everything?
RSE 6 February 2002
Gavin McCance
10
/
23
…Middleware Activities
Fabric and Mass Storage Management
Interfaces to underlying systems
Network Monitoring
What’s the bandwidth from here to there?
Security
Crops up everywhere … transparent to
applications
RSE
6
February
2002
Gavin McCance
11
/
23
Data Management
Data Replication
Meta Data Catalogues
Replica Optimisation
Which replica should I use?
RSE
6
February
2002
Gavin McCance
12
/23
Data Replication
Problems if data exist only in one place
No one site can afford to store all data!
Multiple accesses to the same data
overload network! Petabytes!
What if site / network is down?
Make Replica!
But need to keep track of
all the files and their various replica!
Need replica catalogue!
RSE
6
February
2002
Gavin McCance
13
/
23
Replica Catalogues
Distributed catalogue in database:
Have a globally unique Logical File Name
(LFN) mapping to multiple physical instances
of the file (PFNs).
Database must be globally
accessible and secure
Key is to leverage industry
standard technologies
File
-
1
File
-
1
File
-
1
File
-
1
Paris
Glasgow
Chicago
LFN
RSE
6
February
2002
Gavin McCance
14
/
23
Metadata Catalogue
Allows a client to access securely any remote
SQL database on the Grid over HTTP(S)
= SQL Metadata Service
Oracle
MySQL
PostgreSQL
+
PKI Security
+
Standard communication
protocols
(XML/SOAP over HTTPS)
RSE
6
February
2002
Gavin McCance
15
/
23
Distribution
Don’t want a single point of failure or
bottleneck
Must distribute SQL database
Designing scalable architectures
e.g. a RC may exist on each storage site
responsible for its own files
CERN Root RC
CERN RC
UK RC
INFN RC
Queries will propagate down
until replica information is
found…
RSE
6
February
2002
Gavin McCance
16
/
23
Choosing the ‘best’
What does the ‘best’ replica mean?
Nearest? Fastest? Real cost?
For multiple files, the ‘best’ run location
is some minimisation
Network cost
–
network monitoring
Monetary cost
–
EU
-
US link
A reasonable decision must be made on
the basis of limited information!
RSE 6 February 2002
Gavin McCance
17
/
23
Economic models
Data files viewed as ‘commodities’ to be
bought and sold by storage sites
The ‘buyer’ is a job requesting a file
The (virtual) ‘cost’ is:
Reverse auction, buy from ‘cheapest’
)
Discount(
)
(
Policy
)
(
,
file
p
network
t
C
B
A
B
A
B
A
RSE
6
February
2002
Gavin McCance
18
/
23
Economic replication
If a storage site believes it can ‘make money’ on a
popular file (based on its observation of access
patterns) it can buy it from another site (replication)
Selfish local optimisation should lead to a reasonable
global optimisation for file distribution
Inherently distributed optimisation.. No distributed
planning overhead!
File 1
1
20
14
15
1
A
B
RSE 6 February 2002
Gavin McCance
19
/
23
Will it work???
real Grid...
…simulated Grid provides
testing arena for these ideas!
Developing simulation tool
RSE 6 February 2002
Gavin McCance
20
/
23
Testbed Software
UK HEP is providing testbed
EU experiments.. CERN LHC
US experiments.. Fermilab / SLAC
First EU DataGrid
software release!
Currently being tested..
RSE 6 February 2002
Gavin McCance
21
/
23
Experiment integration
Taking the kit and trying to integrate it
into the experiments’ software
frameworks
ATLAS/LHCb software
framework (GAUDI)
Grid middleware
GANGA framework
Make Grid Services
transparently
available to
ATLAS and LHCb
programs
RSE
6
February
2002
Gavin McCance
22
/
23
UK and EU Testbed
Some successful
tests so far…
e.g. large file
transfers UK, Italy,
US and CERN
Increasing Monte
Carlo challenges
planned
Currently UK testbed
RSE 6 February 2002
Gavin McCance
23
/
23
…finally
Basic Grid software has been delivered
More developments to come
Integration with experiments and testing
Already successful tests
A excellent base to build on!
Plenty still to do!
Enter the password to open this PDF file:
File name:
-
File size:
-
Title:
-
Author:
-
Subject:
-
Keywords:
-
Creation Date:
-
Modification Date:
-
Creator:
-
PDF Producer:
-
PDF Version:
-
Page Count:
-
Preparing document for printing…
0%
Comments 0
Log in to post a comment