Data Intensive Distributed Computing - Center for Computation ...

hornbeastcalmΔιαχείριση Δεδομένων

27 Νοε 2012 (πριν από 4 χρόνια και 11 μήνες)

254 εμφανίσεις

2
7
& ' s -s )s
$ 
 
s s.s) s
/
1.Coordinate “resources that are not subject to
centralized control”
2.Use “standard, open, general-purpose protocols
and interfaces”
3.To deliver “nontrivial qualities of service*,
8
 0 1.ss
s
￿
23& 
￿
423& 
•Homogeneous
•Closely Coupled
•Low Latency
•Single Administrative
Domain
•Heterogeneous
•Loosely Coupled
•High Latency
•Single Administrative
Domain
•Heterogeneous
•Loosely Coupled
•Very High Latency
•Multiple Administrative
Domains
￿Parallel Computing
￿Distributed Computing
￿Grid Computing
Campus
CampusCampus
Campus
Machine RoomMachine RoomMachine RoomMachine Room
NationNationNationNation
9
    sss
$ s /1  
% 5ss   s )s     ))
   
%     6 
$ 7)) ) 
% 5s ss .s s   s  
  8
%   19&:7;69
$;  
% 1)  <  8  < s
% 4/7 s 21 
10
    sss(cont.)
$   &s 
% ss    s   ) .s /  
% = )    .    s
% 7)9 .>).ss?61  1@ s  .
$   /  
% 9 / ) ) A A)   s
% 1) s  ss s  s     )s
11
16 B  > Bs
  ( .       
  s)) .).ss 
  s /   s
A   * 
9  5 
9  
5 ?95@
   
 .   .ss    
  ss )  s ) .
 )s.s * 
;9; 1
9 )1.s
 ?91@
         ss 
s  s(   s 5*1*;9s
 /  s(   ss
s * 
;9; 1
;91
 
' s5# (1 s s2 
12
16 B  > Bs
    .     
 )<  
s * 
21'
28  
9 )< 9*
1   
    .   s   
 .ss)) .  ).ss
0 s
*
;91
> >).ss  
 
           /
 As 0     
)  s(   s
* 
21'
&    C  
    /  .
    .     
 ss  ) 21ss s
* s * 
21
&   > 8 
 
)  .#(     .ss).ss
0s 1 61  &; 111
 ).* 
21'
 >).ss28 
' s5# 31 s 2 
3
13
16 B  > Bs
1     Bs8))
5**
 As * *
5**1
5 
1 
5*1*s s     B 
s  ss D!/3s
  * 
21'
  
' s5# 31 s 2 
14
10,000s processors
PetaBytes of storage
15
)*"'  
  D!/3s
26
24
8
4
HPSS
5
HPSS
HPSS UniTree
External
Networks
External
Networks
External
Networks
External
Networks
Site Resources Site Resources
Site ResourcesSite Resources
NCSA/PACI
8 TF
240 TB
SDSC
4.1 TF
225 TB
Caltech Argonne
www.teragrid.org
16
www.ivdgl.org
C 
&    C      /  .
Tier0/1 facility
Tier2 facility
10 Gbps link
2.5 Gbps link
622 Mbps link
Other link
Tier3 facility
17
s  s
19&:) 
$  . s s ) ) /
 s 
$ 5ss).s   s   .E)
    )s 
;) s
' :)  ')s:) 
$;  !!! !!!   s  s 8) 
s s   ) >
$;   ! ' 3s
%   D! ' 3s 
$  sFG!!HH
%   IF!!6
18
#s )) s
$ 28>  6 s
%  A /s /s)  .
%  A /6>&
%  A /s.s
$ 1 .
$#s  6  
$  )s
$ 28    s & s  
4
19
#  
$      s
% & ' s ( ss 
$    s
% )    *
20
#< s    6  
$    
%#  < . 0s  
$    ss/ s   
% 7)A  /s  
$ 1 ))As   /   
% 9**     0    ss
$ 1 0/   ss
% 9**   / ss ) )     s?7'@      
s  /Bs
$   ' 
% > ss   s s.s/  s 
21
#< s    6  (Cont.)
$ >  s)   0     <ss
   s
$ 6        
%#s  < .   s
% 1)/s       s
$ 1 .
% >    s s.ss
% 1 s    s s
% >  8 / 0s  
$ C   
% s    ./s   s s.s?+   E,@
    
22
'  C8  
  6  
Location based on
data attributes
Location of one or
more physical replicas
State of grid resources,
performance measurements
and predictions
Metadata Service
Application
Replica Location
Service
Information Services
Planner:
Data location,
Replica selection,
Selection of compute
and storage nodes
Security and Policy
Executor:
Initiates
data transfers and
computations
Data Movement
Data Access
Compute Resources
Storage Resources
23
 )  . s
Fabric (e.g., storage, compute nodes, networks)
Connectivity (e.g., TCP/IP, GSI)
Resource: sharing single resources (e.g.,
GridFTP, SRM, DBMS)
General services for coordinating multiple
resources (e.g., RLS, MCS, RFT, Federation,
Brokering)
Services for coordinating multiple resources that
are specific to an application domain or virtual
organization (e.g., Authorization, Consistency,
Workflow)
24
  1 s  s
$ )6      1 ?61@
%    )  ss  ss   /s?   @)  s /
  s8)   s   s
$ )#    1 ?# 1@
% s / s .   s   s  .J  s  s
/8   s    ).s    s   s
$ ) '>    s    
% 90s/ s       s s s  s s 
 s . ) A . s s    >/ sEs
$ )# /' s ?#'@s 
%  s ?0s  8/s @)   ss  / 
s  s s s /   .  s      s
5
25
# 6   s
$   s   s
% >  /.s > /.s  
$#       s
%'   
% >     8     s  s 
) /  
$ &sss
%    s s s
%   8  s
% 1  /.
%# /.
26
LRC
LRC
LRC
RLI
RLI
LRC
LRC
Replica Location Indexes
Local Replica Catalogs
• LRCs contain consistent information about logical-to-
target mappings on a site
• RLIs nodes aggregate information about LRCs
• Soft state updates from LRCs to RLIs: relaxed
consistency of index information, used to rebuild index
after failures
• Arbitrary levels of RLI hierarchy
27
LRC
LRC
LRC
RLI
RLI
LRC
LRC
Replica Location Indexes
Local Replica Catalogs
# 18)2 # . 
>   &0/.1 1s
28
# 18)# .