Experiences of the Grid - GridPP

obtainablerabbiΔιαχείριση Δεδομένων

31 Ιαν 2013 (πριν από 4 χρόνια και 6 μήνες)

120 εμφανίσεις

Experiences of the Grid…

Gavin McCance

University of Glasgow

NeSC Meeting, 24 October 2001

2
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

Background

Experimental Particle Physics
background


Analysing the structure of matter


…Fortran (19)77 !


Working in ‘Grid’
-
like areas since
January this year

3
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

GridPP

20+ institutes…

4
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

…GridPP

£17M 3
-
year project

Working in collaboration with EU
DataGrid project


Middleware production

Integration of middleware technologies
into HEP experiments

Validation of Grid Software

5
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

…GridPP

Initial GridPP testbed underway


A personal snapshot of activities on
the grid…


Middleware activities we’re involved in


Some examples


Technologies we’re using


Issues with integration of ‘Grid’ with
particle physics experiments

6
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

Middleware

What is middleware…???

Application programs


local gridopen()

Data access specifics


HPSS, Castor

Job submission specifics


PBS, LSF

Specific security procedures

Grid middleware

Layered API’s.

Transparent security.


Transparent data access.


Intelligent use of

distributed resources.

7
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

Middleware Activities

GridPP ~mirrors EU DataGrid:

Workload Management


What jobs go where?

Data Management (*)


Where’s the (best) data?

Information Services (*)


What’s the state of everything?

8
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

…Middleware Activities

Fabric Management


Interfaces to underlying systems

Mass Storage Management


How to get the data to/from the fabric e.g.
Implementing ‘file
-
save()’ APIs for different mass
storage systems

Security


Crops up everywhere … transparent to
applications

9
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

Data Management

Data Replication

Transparent and Secure Data Access

Meta Data Storage

Query Optimisation


10
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

Example problem:

Data Replication

Problems if data exist only in one place


Multiple accesses to the same data
overload network! Petabytes!


Funding constraints! e.g. CERN can’t store
all of the data required

Make Replica!

But need to keep track of
all the files and their various replica!


Need replica catalogue!

11
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

…Catalogues

Examples solutions:

Have a globally unique Logical File Name
(LFN) mapping to multiple physical instances
of the file (PFNs).


Replica selection required


Choose the
‘best’

/ ‘nearest’ / ‘fastest’


Cost modelling… how time expensive to transfer
files X’ from A to B

File
-
1

File
-
1

File
-
1

File
-
1

Paris

Glasgow

Chicago

LFN

12
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

…Data Replication

Grid Data Mirroring Package


C, C++, JAVA, command
-
line APIs

Replication issues:


File transfer…


Synchronisation / consistency models


Basic middleware doesn’t enforce any policy


Scalable architectures

13
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

…GDMP

File transfer uses GridFTP


Existing

IETF
-
approved (?RFC?) ftp additions


+ the
standard grid security

(GSI)

Registers new files in replica catalogue


E.g. interfaced to the
existing

Globus Replica
Catalogue

Basic replica manager functionality to
maintain consistency of replica sets

14
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

…Implementation issues

Structure not imposed by the
middleware software itself…


But … must think about scalable
implementations

E.g. a RC may exist on each storage
element


responsible for its own files


CERN Root RC

CERN RC

UK RC

INFN RC

Queries will propagate down
until replica information is
found…

15
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

…Longer term problems

Query / Replica Optimisation


Grid can make / delete replica


Eg. Many people in Glasgow & Edinburgh
access the ATLAS Higgs dataset ‘A1’…


Autonomously make new replica in / near Scotland
based on historical information


Grid might re
-
cluster data

A2

A3

A1

B1

B3

B2

Glasgow

Paris

A2

A3

A1

B1

B3

B2

Glasgow

Paris

16
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

…longer term

real Grid...

…simulated Grid provides
testing arena for more
adventurous ideas!

MONARC simulation tool

17
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

…Integration of middleware

Many iterations of requirements and
use
-
cases with end
-
users… meetings…

Middleware solutions must be scalable
and useable by a variety of end users


HEP, Biological, Earth sciences, Astro

Always looking for common elements


E.g. replica / meta
-
data catalogues… data
transport… security…

18
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

…examples of common interfaces:

generic meta
-
data catalogue tools

SQL Database Service:


Problem: many relational databases,
diverse security, diverse wire protocols


…Solution:


Build on existing wire protocols: XML
transported over HTTP(S)


Grid standard security framework (GSI)


19
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

..examples

Leverage open
-
source technology


JAVA servlet based (Apache Tomcat
engine)


JDBC drivers


Utilises Oracle’s XSQL servlet (open source)


Security over HTTPS with
Grid
-
standard

GSI mechanism

20
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

…examples






Allows
any HTTP compliant

system e.g. Web
-
browsers / standard C++ HTTP libraries to access
any

relational database…

= SQL Database Service (Spitfire)

Oracle

PostgreSQL

+

PKI Security

+

Standard communication

protocols

(XML over HTTPS)

21
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

Global Grid Forum

Global Grid Forum meetings


GGF1
: Amsterdam meeting in April 2001

Helps define aspects common to all
Grid
-
like projects.


E.g. architectures, ‘grid’ protocols


As example… Grid Monitoring
Architecture (GMA)

22
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

Information Services
-

GMA

One Implementation of the GMA




Globus MDS, currently based on
(Open)LDAP


Hierarchical directory like structure


Very fast for information retrieval if you already
know the query


designed into structure.


Bad for complex or ranged queries

23
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

..complementary implementation

Producer

Producer
API

Register,

re
-
register,

publish

Relational
Database

Consumer

Querying
API

Query

Producer

Servlet

Registry

Servlet

Schema

Servlet

subscribe

stream

Implementation of GMA


Relational queries in
SQL format

24
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

…relational GMA

Information is transferred in generic
SQL format…

‘Producers’

of information
register

themselves…

‘Consumers’

construct (possibly
complex) SQL query and are streamed
query results directly from Producers.

25
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

…implementation

Again, uses JAVA servlets


Tomcat servlet engine

Again, communication with servlet is
over
standard HTTP.


All the internal parts communicate via
HTTP

and
XML



modular design,
easily replaceable…

26
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

Useful Tools…

JAVA
… nicely platform independent

UML

Universal(?) Modelling Language


Architecture and API’s ‘should be’ defined
in this…!

CASE tools


Together Control Centre

27
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

…useful tools

Globus

toolkit


Both the original and its java implementation
(CoG)


My experience of
CoG

so far is generally
good…!


Easy GSI authentication, Globus file transfer,
Globus job submission, MDS interface

28
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

Testbeds

For GridPP, primary testbeds are the HEP
experiment ones



CERN LHC (EU DataGrid WP8)


US experiments, e.g. Fermilab, SLAC


First software release now!!


Integration team ‘show
-
and
-
tell’ at CERN end of
this month…

29
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

...testbed work

Grid software
packaged for
release to
experiments!

Primarily packaged
using RPM

For end of October release, supported platforms are:

Linux

(and
Solaris

on a best effort basis)

30
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

..Globus installation

Generally found the Globus software
installation OK!


Successfully deployed on a number of batch
systems in UK


Experience fed back into eScience Centres

Difficulties were setting up and recognising
each countries’ Certificate Authorities (CAs)



Tricky legal implications to resolve!

31
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

Testbed work so far…

UK Certificate Authority set
-
up…


Many institutes already on testbed

Grid Status and Network monitoring
demonstrator available soon

Networking status
information provided
by GridPP and
DataGrid networking
groups!

32
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

…testbed work so far

Successful tests within ATLAS (and
others) of some middleware products


E.g. Large
file transfers

between UK, Italy,
US and CERN


Further tests planned with new release!

33
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

…experimental integration

Work to do…


Taking the kit and trying to integrate it into
the experiments’ software frameworks

ATLAS/LHCb software

framework (GAUDI)

Grid middleware

GANGA framework

Make Grid Services


transparently

available to

ATLAS and LHCb

programs

34
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

Grid validation

Preliminary tests of basic middleware
has been successful



Now we have opportunity to see how it
performs and scales with real datasets
and real experimental users

35
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

Overall experience

Middleware development is fun!


Several good products have already been
delivered


Re
-
using
industry standard components and
protocols

where they exist


LDAP, SQL, HTTP(S), XML, SOAP


PKI security


Open Source…!

36
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

…overall

Middleware being built using a variety
of languages…
JAVA, C++, C, Python


APIs should be available for all JAVA,
C++, C and command line… web
access(?)

37
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

…overall

Coordination very important

Forums for discussion:





Vital to ensure middleware is useful to a wide
range of applications

Prevent divergent technology

38
/38

NeSC 24 October 2001

Gavin McCance, University of Glasgow

…finally

Preliminary grid software architectures
have been defined

Basic middleware has been delivered

Large scale validation underway NOW!!


A excellent base to build on!


Plenty still to do!