edms_tmp_2218212_Applications - EU DataGrid Tutorial - CERN

caddiepastData Management

Jan 31, 2013 (4 years and 4 months ago)

316 views

EDG Application



The European DataGrid Project Team

http://www.eu
-
datagrid.org

Using The EDG Testbed


GGF 5
-

n
°

2

Contents




Use Cases


High Energy Physics


Earth Observation


Biomedical Applications


Using The EDG Testbed


GGF 5
-

n
°

3

EDG Application Areas


High Energy Physics

Biomedical
Applications


Earth Observation

Science Applications

Using The EDG Testbed


GGF 5
-

n
°

4

High Energy Physics


4 Experiments on LHC

CMS

ATLAS

LHC
b

~6
-
8 PetaBytes / year

~10
8

events/year

~10
3

batch and interactive users


Using The EDG Testbed


GGF 5
-

n
°

5

Europe: 267 institutes, 4603 users

Elsewhere: 208 institutes, 1632 users

CERN’s Network in the World

Using The EDG Testbed


GGF 5
-

n
°

6

Data Flow in LHC


RAW D
ata

DAQ

Trigger

Reconstruction

Event Summary Data (ESD)

Reconstruction Tags

RAW Tags

Conditions / Calibration
Data


Physics Generator

Detector Simulation

Generator Data

RAWmc Data

Monte Carlo

Reconstruction

Event Summary Data (ESD)

Reconstruct
ion Tags

RAWmc Tags

Conditions /
Calibration Data

Using The EDG Testbed


GGF 5
-

n
°

7

LHCb EDG Integration


LHCb


LHCb distributed computing environment


Integration of DataGrid middleware


Authentication


Job submission to DataGrid


Monitoring and control


Data replication


Resource scheduling


use of CERN MSS

Using The EDG Testbed


GGF 5
-

n
°

8

LHCb


LHC collider experiment


10
9
events * 1Mb = 1 Pb


Need a distributed model


Create, distribute and keep track of data automatically


Using The EDG Testbed


GGF 5
-

n
°

9

Transfer data to

Mass store

Submit jobs

remotely

viaWeb

Data Quality

Check

Execute

on farm

LHCb distributed computing environment

Update bookkeeping

database

Using The EDG Testbed


GGF 5
-

n
°

10

Submit jobs

remotely

via Web

Execute

on farm

Monitor

performance

of farm via

Web

Update bookkeeping

database

Transfer data to

CASTOR (and
HPSS, RAL
Datastore)

Data Quality

Check ‘Online’


UserInterface



WMS



Information

Services



Replica
Management



Online histogram
production using
GRID pipes



MetaData

Catalog


LHCb Environment using EDG
Middleware

Using The EDG Testbed


GGF 5
-

n
°

11

1. Authentication



grid
-
proxy
-
init

Using The EDG Testbed


GGF 5
-

n
°

12

2. Job Submission


dg
-
job
-
submit /home/evh/sicb/sicb/bbincl1600061.jdl
-
o
/home/evh/logsub

bbincl1600061.jdl:

#

Executable = "script_prod";

Arguments = "1600061,v235r4dst,v233r2";

StdOutput = "file1600061.output";

StdError = "file1600061.err";

InputSandbox =
{"/home/evhtbed/scripts/x509up_u149","/home/evhtbed/sicb/mcsend","/ho
me/evhtbed/sicb/fsize","/home/evhtbed/sicb/cdispose.class","/home/evh
tbed/v235r4dst.tar.gz","/home/evhtbed/sicb/sicb/bbincl1600061.sh","/h
ome/evhtbed/script_prod","/home/evhtbed/sicb/sicb1600061.dat","/home/
evhtbed/sicb/sicb1600062.dat","/home/evhtbed/sicb/sicb1600063.dat","/
home/evhtbed/v233r2.tar.gz"};

OutputSandbox =
{"job1600061.txt","D1600063","file1600061.output","file1600061.err","
job1600062.txt","job1600063.txt"};


Using The EDG Testbed


GGF 5
-

n
°

13

3. Monitoring and Control



dg
-
job
-
status



dg
-
job
-
cancel



dg
-
job
-
get
-
output

Using The EDG Testbed


GGF 5
-

n
°

14

3. Monitoring and Control

Using The EDG Testbed


GGF 5
-

n
°

15

3. Monitoring and Control

Using The EDG Testbed


GGF 5
-

n
°

16

3. Monitoring and Control

Using The EDG Testbed


GGF 5
-

n
°

17

3. Monitoring and Control

Using The EDG Testbed


GGF 5
-

n
°

18

Job

data

Local disk

Compute Element

data

Mass store

replica

catalog
(Nikhef)

data

globus
-
url
-
copy

rfcp

Job

data

Storage Element

replica
-
get

publish

register
-
local
-
file

Storage Element

globus
-
url
-
copy

Using The EDG Testbed


GGF 5
-

n
°

19

4. Publish data on storage element



Copy data file to storage element:

globus
-
url
-
copy
file:///${chemin}/L69999

\



gsiftp://lxshare0219.cern.ch/flatfiles/SE1/lhcb/L69999



Register stored data in the catalog:

/opt/globus/bin/globus
-
job
-
run lxshare0219.cern.ch
\

/bin/bash
-
c "export
GDMP_CONFIG_FILE=/opt/edg/lhcb/etc/gdmp.conf;
\

/opt/edg/bin/gdmp_register_local_file
-
d /flatfiles/SE1/lhcb"



Publish catalog:

/opt/globus/bin/globus
-
job
-
run lxshare0219.cern.ch
\

/bin/bash
-
c "export
GDMP_CONFIG_FILE=/opt/edg/lhcb/etc/gdmp.conf;
\

/opt/edg/bin/gdmp_publish_catalogue
-
n"

Using The EDG Testbed


GGF 5
-

n
°

20

The ALICE Event

Using The EDG Testbed


GGF 5
-

n
°

21

The ALICE Event Cont’d


##
-----

Job Description for Aliroot
-----


## author:
roberto.barbera@ct.infn.it

Executable = "/bin/sh";

StdOutput = "aliroot.out";

StdError = "aliroot.err";

InputSandbox =
{"start_aliroot.sh","rootrc","grun.C","Config.C"};

OutputSandbox =
{"aliroot.err","aliroot.out","galice.root"};

RetryCount = 7;

Arguments = "start_aliroot.sh 3.02.04 3.07.01";

Requirements =
Member(other.RunTimeEnvironment,"ALICE
-
3.07.01");

( start_aliroot.sh) :

#!/bin/sh

mv rootrc $HOME/.rootrc

echo "ALICE_ROOT_DIR is set to: $ALICE_ROOT_DIR"

export ROOTSYS=$ALICE_ROOT_DIR/root/$1

export PATH=$PATH:$ROOTSYS/bin

export
LD_LIBRARY_PATH=$ROOTSYS/lib:$LD_LIBRARY_PATH

export ALICE=$ALICE_ROOT_DIR/aliroot

export ALICE_LEVEL=$2

export ALICE_ROOT=$ALICE/$ALICE_LEVEL

export ALICE_TARGET=`uname`

export
LD_LIBRARY_PATH=$ALICE_ROOT/lib/tgt_$ALICE_TARGET:$
LD_LIBRARY_PATH

export
PATH=$PATH:$ALICE_ROOT/bin/tgt_$ALICE_TARGET:$ALIC
E_ROOT/share

export MANPATH=$MANPATH:$ALICE_ROOT/man

$ALICE_ROOT/bin/tgt_$ALICE_TARGET/aliroot
-
q
-
b grun.C


Using The EDG Testbed


GGF 5
-

n
°

22

Earth Observation
Application

Processing of raw GOME data

to ozone profiles

With OPERA (KNMI)

Validate GOME


ozone profiles with


Ground Based measurements


(IPSL)

Raw satellite data


from the GOME instrument


(ESA)

2 different jobs are executed on the
TESTBED
, using data provided
via the sandbox model

Visualization

LIDAR


data

Using The EDG Testbed


GGF 5
-

n
°

23

OPERA application (KNMI)


From wave spectra measured by the GOME instrument on

the
ERS satellite ozone profiles can be calculated. ESA provides
these spectra as level 1 data. This level 1 data is then
processed using OPERA to produce ozone profiles, a level 2
product. The algorithm and s/w (OPERA) are developed by
KNMI.


GOME takes ~30.000 usable
measurements for ozone
profile retrieval per day.

The calculation of 1 profile
takes ~2 min on a 800Mhz
PIII.

One day of profiles will take
40 days on 1 computer.

Using The EDG Testbed


GGF 5
-

n
°

24

Validation application (IPSL)


Produced profiles by OPERA are validated by
IPSL using ground based LIDAR
measurements.


Since the LIDAR data are in
-
situ, pre
-
selection of the global GOME data has to be
performed to create a dataset which is
geographically and temporally in coincidence.


The main function of the program is to
perform statistical operations like the bias
between GOME and LIDAR data for
different altitudes and its standard
deviations.


The output of the validation program are 2
plots, generated by xmgr.

Using The EDG Testbed


GGF 5
-

n
°

25

Validation Output

Figure

1
:



Estimation of the bias
between Gome and Lidar
using one month of data.

Figure

2

:



example of 2 profiles :
Comparison between
Gome profile and lidar
profile for the 2nd
October 2000.

Using The EDG Testbed


GGF 5
-

n
°

26

Ozone Data Processing and
Validation using the EDG Testbed

Completed the adaptation of 3 different EO Applications (OPERA,
NOPREGO, Validation) to run on EDG

Demonstrated Ozone data processing and validation across three sites
for 3 months of observations in a coincident area:

1.
Transfer
Level1

(raw) data to the Grid Storage Element

2.
Register
Level1

data with the Replica Manager

3.
Submit jobs to process the
Level1

data, produce
Level2

data products

Jobs running on the CEs locate
Leve1

data by using the BrokerInfoAPI

1.
Repeat step 1
-
3 for level 2 products

2.
Retrieve validation results and visualize at the User Interface

Using The EDG Testbed


GGF 5
-

n
°

27

Demonstration on Testbed 1

User

Interface

Replica

Manager

Submit job

input

data

Mydata

input

data

input

data

input

data

CE

SE

Site H

CE

SE

Site G

CE

SE

Site F

CE

SE

Site E

CE

SE

Site D

CE

SE

Site C

CE

SE

Site B

Replica

Catalog

Replicate

Data

MetaData

Step 1:

Transfer
Level1

data to the Grid Storage
Element

Step 2:

Register
Level1

data with the Replica Manager
(replicate if necessary)

User

Using The EDG Testbed


GGF 5
-

n
°

28

Demonstration on Testbed 1

Resource

Broker

CE

SE

Site H

CE

SE

Site G

CE

SE

Site F

CE

SE

Site E

CE

SE

Site D

CE

SE

Site C

CE

SE

Site B

Certificate

Authorities

User

Interface

JDL

script

Executable

Myjob

Submit job

Request
status

Check
certificate

Search

Information

Index

Replica

Catalog

Search

input

data

input

data

input

data

LFN

LFN

LFN

Retrieve result

LFN

PFN

Logical filename

Physical filename

LFN

PFN

::

LFN

PFN

::

LFN

PFN

::

User

Step 3:

Submit jobs to process
Level1

data, to produce
Level2

data

Step 4:

Transfer
Level2

data products to the
Storage Element

Using The EDG Testbed


GGF 5
-

n
°

29

Validation of two Different Ozone
Processing Algorithms vs Lidar Profiles

Step 5:

Retrieve and
visualise results

Using The EDG Testbed


GGF 5
-

n
°

30

Used JDL file




Executable

= "o3gome
-
lidar_xmgr.final";

StdOutput

= "appli.out";

StdError

= "appli.err";

InputSandbox

= {"/home/leroy/DEMO_190202/o3gome
-
lidar_xmgr.final",

"/home/leroy/DEMO_190202/obs20001019.dat",

"/home/leroy/DEMO_190202/obs20001002.dat",

"/home/leroy/DEMO_190202/obs20001003.dat",

"/home/leroy/DEMO_190202/obs20001004.dat",

"/home/leroy/DEMO_190202/obs20001005.dat",

"/home/leroy/DEMO_190202/obs20001006.dat",

"/home/leroy/DEMO_190202/select_coinc.exe",

"/home/leroy/DEMO_190202/data_process_demoxmgr",

"/home/leroy/DEMO_190202/oho30010.gol"};

OutputSandbox


=
{"out_proc.dat","profil
_
gome.dat","profil_lidar.dat",

"appli.out","appli.err"};

Requirements

= other.
OpSys == “RH 6.2”
;

RetryCount

= 10;

Rank

= other.MaxCpuTime;






The produced profiles by OPERA are validated by IPSL using ground based
LIDAR measurements.


One Month of data (gome and lidar data) is used to do a analysis between
the different measurements


The result is visualized using xmgr.

Using The EDG Testbed


GGF 5
-

n
°

31

World
-
Wide Ozone Distribution
Mapping

Need for systematic and

global mapping of

ozone distribution

Large amount of information

about atmosphere gases

stored in Terabytes of data

GOME

SCIAMACHY

Scientific community
: need for a

collaborative environment to study

problems such as ozone depletion

GRID

Using The EDG Testbed


GGF 5
-

n
°

34

EO Web Portal Prototype

User

GRID
Portal

Using GRID Portal user
selects world region
and time frame to be
processed

Level 1
Catalogue

Queries L1
Catalogue
and
retrieves
orbit
number and
logical name
file

Queries L2
Catalogue to
check if the
data was
already
processed

Level1 Orbit Data

Querying metadata

Level2 Pixel Data

Using The EDG Testbed


GGF 5
-

n
°

35

EO Web Portal Prototype

GRID

Computing

Element

Storage
Elements

GRID
Portal

Level 2
Catalogue

Level 2 Catalogue will give
the logical file names.

Queries Replica Catalogue to
retrieve the path of our logical file
names.

If the data already exists …

Replica
Catalogue

Using The EDG Testbed


GGF 5
-

n
°

36

EO Web Portal Prototype

GRID
Portal

Level 1
Catalogue

Level 1 Catalogue will
give the orbit number*

Level 1

Storage (AMS)

Retrieve Level
1 Data orbit
file, extract
the requested
geographic
area

GRID

Computing

Element

Storage
Elements

Places Level 1 Data into
the storage element

Launches the
Job in the
Computing
Element

When
completed
announces
a new level
2 data into
the
catalogue

If data was not yet processed…

Level 2
Catalogue

Using The EDG Testbed


GGF 5
-

n
°

37

Biomedical Applications on v1.1

Goals: collect biomedical requirements and deploy first applications on the
testbed

Biomedical encompass genomics, post
-
genomics and biomedical applications









Challenges

The biomedical community has NO strong center of gravity in Europe

Very large user community with little computer science awarness

Complex needs (parallel applications, data security, metadata...)

Using The EDG Testbed


GGF 5
-

n
°

38

BLAST: Basic Local Alignment Search
Tool

BLAST is the first step for analysing new sequences: to compare DNA
or protein sequences to other ones stored in personal or public
databases.

BLAST is costly and a good candidate for gridification:

Requires equipment to store databases and run algorithms

Requires manpower for system & network maintenance and frequent update
of databases

More and more biologists...

... compare larger and larger sequences (whole genomes)...

... to more and more genomes...

... with fancier and fancier algorithms !

Using The EDG Testbed


GGF 5
-

n
°

39

The visual DataGrid BLAST

Most biologists use integrated web portals for their genomics comparative
analysis: no need to worry about the biological file format and the method
arguments.

The vDG BLAST includes:

a graphical interface to enter query

sequences and select the reference

database

A script to execute the BLAST

algorithm on the grid

A graphical interface to

analyse results


Using The EDG Testbed


GGF 5
-

n
°

40

Running BLAST on the grid

Seq
1

>

dcscdssdcsdcdsc

bscdsbcbjbfvbfvbvfbvb
vbhvbhsvbhdvbhfdbvfd


Seq
2

>

bvdfvfdvhbdfvb

bhvdsvbhvbhdvrefghefg
dscgdfgcsdycgdkcsqkc





Seqn

>

bvdfvfdvhbdfvb

bhvdsvbhvbhdvrefghefg
dscgdfgcsdycgdkcsqkch
dsqhfduhdhdhqedezhhe
zldhezhfehflezfzejfv

DB

BLAST

dedze
dzdze
dezdze
cdscsd
cscdss
dcsdcd
scbscd
sbcbjb
f

dedzed
zdzede
zdzecd
scsdcs
cdssdc
sdcdsc
bscdsb
cbjbf

dedze
dzdzed
ezdzec
dscsdc
scdssd
csdcds
cbscds
bcbjbf

dedze
dzdze
dezdze
cdscsd
cscdss
dcsdcd
scbscd
sbcbjb
f

DB

BLAST

dedze
dzdze
dezdze
cdscsd
cscdss
dcsdcd
scbscd
sbcbjb
f

dedze
dzdze
dezdze
cdscsd
cscdss
dcsdcd
scbscd
sbcbjb
f

DB

BLAST

dedze
dzdze
dezdz
ecdsc
sdcsc
dssdc
sdcds
cbscd
sbcbj
bf

dedze
dzdze
dezdz
ecdsc
sdcsc
dssdc
sdcds
cbscd
sbcbj
bf

dedze
dzdze
dezdz
ecdsc
sdcsc
dssdc
sdcds
cbscd
sbcbj
bf

dedze
dzdze
dezdz
ecdsc
sdcsc
dssdc
sdcds
cbscd
sbcbj
bf

dedze
dzdze
dezdz
ecdsc
sdcsc
dssdc
sdcds
cbscd
sbcbj
bf

dedze
dzdze
dezdz
ecdsc
sdcsc
dssdc
sdcds
cbscd
sbcbj
bf

dedze
dzdze
dezdz
ecdsc
sdcsc
dssdc
sdcds
cbscd
sbcbj
bf

dedze
dzdze
dezdz
ecdsc
sdcsc
dssdc
sdcds
cbscd
sbcbj
bf

dedze
dzdze
dezdz
ecdsc
sdcsc
dssdc
sdcds
cbscd
sbcbj
bf

dedze
dzdze
dezdz
ecdsc
sdcsc
dssdc
sdcds
cbscd
sbcbj
bf

dedze
dzdze
dezdz
ecdsc
sdcsc
dssdc
sdcds
cbscd
sbcbj
bf

dedze
dzdze
dezdz
ecdsc
sdcsc
dssdc
sdcds
cbscd
sbcbj
bf


RESULT


dedzedzdzedezdzecdscsdc
scdssdcsdcdscbscdsbcbjbf
vbfvbvfbvbvbhvbhsvbhdvbh
fdbvfdbvdfvfdvhbdfvbhdb
hvdsvbhvbhdvrefghefgdscg
dfgcsdycgdkcsqkcqhdsqhfd
uhdhdhqedezhdhezldhezhf
ehflezfzeflehfhezfhehfezh
flezhflhfhfelhfehflzlhfzdja
zslzdhfhfdfezhfehfizhflqf
hduhsdslchlkchudcscscdscd
scdscsddzdzeqvnvqvnq!
Vqlvkndlkvnldwdfbwdfbdbd
wdfbfbndblnblkdnblkdbdfb
wfdbfn

dedze
dzdze
dezdze
cdscsd
cscdss
dcsdcd
scbscd
sbcbjb
f

dedze
dzdze
dezdze
cdscsd
cscdss
dcsdcd
scbscd
sbcbjb
f

dedzed
zdzede
zdzecd
scsdcs
cdssdc
sdcdsc
bscdsb
cbjbf

dedze
dzdze
dezdze
cdscsd
cscdss
dcsdcd
scbscd
sbcbjb
f

dedze
dzdze
dezdze
cdscsd
cscdss
dcsdcd
scbscd
sbcbjb
f

dedze
dzdze
dezdze
cdscsd
cscdss
dcsdcd
scbscd
sbcbjb
f

dedze
dzdze
dezdze
cdscsd
cscdss
dcsdcd
scbscd
sbcbjb
f

dedze
dzdze
dezdze
cdscsd
cscdss
dcsdcd
scbscd
sbcbjb
f

dedzed
zdzede
zdzecd
scsdcs
cdssdc
sdcdsc
bscdsb
cbjbf

dedze
dzdze
dezdze
cdscsd
cscdss
dcsdcd
scbscd
sbcbjb
f

dedze
dzdze
dezdze
cdscsd
cscdss
dcsdcd
scbscd
sbcbjb
f

dedze
dzdze
dezdze
cdscsd
cscdss
dcsdcd
scbscd
sbcbjb
f

dedze
dzdze
dezdz
ecdscs
dcscds
sdcsdc
dscbsc
dsbcbj
bf

dedze
dzdze
dezdze
cdscsd
cscdss
dcsdcd
scbscd
sbcbjb
f

dedze
dzdze
dezdze
cdscsd
cscdss
dcsdcd
scbscd
sbcbjb
f

dedzed
zdzede
zdzecd
scsdcs
cdssdc
sdcdsc
bscdsb
cbjbf

dedze
dzdze
dezdze
cdscsd
cscdss
dcsdcd
scbscd
sbcbjb
f

dedze
dzdze
dezdze
cdscsd
cscdss
dcsdcd
scbscd
sbcbjb
f

UI

Computing


element

Input

file

Computing


element

dedze
dzdze
dezdze
cdscsd
cscdss
dcsdcd
scbscd
sbcbjb
f

dedze
dzdze
dezdze
cdscsd
cscdss
dcsdcd
scbscd
sbcbjb
f

dedzed
zdzede
zdzecd
scsdcs
cdssdc
sdcdsc
bscdsb
cbjbf

dedze
dzdze
dezdze
cdscsd
cscdss
dcsdcd
scbscd
sbcbjb
f

dedze
dzdze
dezdze
cdscsd
cscdss
dcsdcd
scbscd
sbcbjb
f

dedze
dzdze
dezdze
cdscsd
cscdss
dcsdcd
scbscd
sbcbjb
f

dedze
dzdze
dezdz
ecdscs
dcscds
sdcsdc
dscbsc
dsbcbj
bf

dedze
dzdze
dezdze
cdscsd
cscdss
dcsdcd
scbscd
sbcbjb
f

dedze
dzdze
dezdze
cdscsd
cscdss
dcsdcd
scbscd
sbcbjb
f

dedze
dzdze
dezdze
cdscsd
cscdss
dcsdcd
scbscd
sbcbjb
f

dedze
dzdze
dezdze
cdscsd
cscdss
dcsdcd
scbscd
sbcbjb
f

Seq1 >

dedzedzdzede
zdzecdscsdcsc
dssdcsdcdscb
scdsbcbjbdfn
dfjvbndfbnbn
fbjnbjxbnxbjk
:nxbf

dedze
dzdze
dezdze
cdscsd
cscdss
dcsdcd
scbscd
sbcbjb
f

dedze
dzdze
dezdze
cdscsd
cscdss
dcsdcd
scbscd
sbcbjb
f

dedzed
zdzede
zdzecd
scsdcs
cdssdc
sdcdsc
bscdsb
cbjbf

Seq2 >

dedzedzdzede
zdzecdscsdcsc
dssdcsdcdscb
scdsbcbjbdfn
dfjvbndfbnbn
fbjnbjxbnxbjk
:nxbf

dedzed
zdzede
zdzecd
scsdcs
cdssdc
sdcdsc
bscdsb
cbjbf

Seqn >

dedzedzdzede
zdzecdscsdcsc
dssdcsdcdscb
scdsbcbjbdfn
dfjvbndfbnbn
fbjnbjxbnxbjk
:nxbf

Using The EDG Testbed


GGF 5
-

n
°

41

Computation time

Swissprot vs Swissprot (100000 sequences)

Running time on one CPU : 228 hours

Tests at Institut de Biologie et Chimie des Protéines (quadboard) :


49 hours

Tests on DataGrid testbed1 (cc
-
in2p3) : 3 hours

Impacts :

Reduced pressure on local computing

Ability to handle very large jobs

Using The EDG Testbed


GGF 5
-

n
°

42

Perspectives for Biomedical
Applications

Grids open new perspectives in large scale genomics analysis

Complete genome annotation

Cross
-
genomes analysis

Data mining on distributed databases

Pipelining of huge automatic bio
-
informatics analysis


Medical image processing

Large databases processing

Anatomy and physiology modeling

Epidemiological studies


Using The EDG Testbed


GGF 5
-

n
°

46

The Future from Biomedical App.
Point of View

Distributed Algorithms
. New distributed "grid
-
aware" algorithms (c.f.
this afternoon's demonstration).

Grid Service Portals
. Service providers taking advantage of the
DataGrid computational power and storage capacity.

Cooperative Framework
. Use the DataGrid as a cooperative framework
for sharing resources, algorithms, and organize experiments in a
cooperative manner.

Cooperative Framework

Grid Service Portals

Distributed Algorithms

EDG Middleware

Biomed

applications

Using The EDG Testbed


GGF 5
-

n
°

47

Parallel Processing

Magnetic Resonance Images simulation using the grid







3 levels of parallelism:

Parallel isochromat computations

Multi
-
slice MRI computation

Parallel magnetization kernel

Magnetisation

computation

kernel

Reconstruction

algorithm

MRI

Image

Virtual

object

MRI

sequence

Using The EDG Testbed


GGF 5
-

n
°

48

Data, Meta
-
Data and Security

Using The EDG Testbed


GGF 5
-

n
°

49

Data
-
processing coupling

Pipeline issues

one pipeline for multiple inputs

load balancing / synchronization

failure / retrial paradigm, logging

dynamic extension of the processed image set

Algorithm DB

Log data transformations

Rebuild transformed data from raw data

Step 5

Step 4

Step2

Step1

Input
dataset

Output images

Step3

Using The EDG Testbed


GGF 5
-

n
°

50

Summary




Use Cases


High Energy Physics


Earth Observation


Biomedical Applications

Using The EDG Testbed


GGF 5
-

n
°

51

Further Information



High Energy Physics

http://datagrid
-
wp8.web.cern.ch/DataGrid
-
WP8/



Bio
-
Informatics

http://marianne.in2p3.fr/datagrid/wp10/index.html



Earth Observation

http://styx.esrin.esa.it/grid/