ppt

arghtalentData Management

Jan 31, 2013 (4 years and 11 months ago)

132 views

GGF Toronto 19.02.02



Spitfire

A Relational DB Service for the Grid

Peter Z. Kunszt

European DataGrid Data Management

CERN Database Group

GGF Toronto 19.02.02

Motivation


Small and large Grid applications


working together to provide access to and management of
massive amounts of data


Examples


replica metadata catalog, service registry, application
metadata, logging and performance monitors


Use


easy to use, interoperable and high
-
performing database
technology


Otherwise


continue to use many proprietary approaches towards
metadata storage and retrieval


GGF Toronto 19.02.02

Baseline


Aim: U
nified Grid enabled front end to relational
databases.


C
onvenient, scalable and efficient storage, retrieval
and query of data held in
any

type of local or remote
RDBMS.


C
ore SQL functionality is
insert
,
delete
,
update

and
query
.


Implement as Grid Service


GGF Toronto 19.02.02

WP
2
: What did we sign up for?

Metadata Management


'Simple' Grid Persistency


Grid Metadata


Application Metadata


Metadata Replication and Consistency


Publish information on the metadata service



GGF Toronto
19.02.02

Collective Services

Information &
Monitoring

Replica Manager

Grid Scheduler

Replica Optimization

Replica Catalog Interface

Grid Application Layer

Job Management

Local Application

Local Database

Fabric services

Configuration

Management

Node

Installation &

Management

Monitoring

and

Fault Tolerance

Resource
Management

Fabric Storage

Management


Grid

Fabric

Local Computing

Grid

Data Management

Metadata
Management

Object to File
Mapper

Underlying Grid Services

Computing
Element
Services

Authorisation,
Authentication
and Accounting

Replica
Catalog

Storage
Element
Services

Spitfire
MetaData
Service

Service
Index

GGF Toronto 19.02.02

Spitfire Architecture

Oracle

DB
2

PostGres

MySQL


Atomic RDBMS is always
consistent


No local replication of data


Role
-
based authorization



Web/Grid Services Paradigm


SOAP interfaces


JDBC interface to RDBMS


But not a 'real' distributed DBMS


no distributed locking &
transactions (yet?)


lazy consistency model


OracleLayer

OracleLayer

PGLayer

MyLayer

Local Spitfire

Layer

Connecting Layer

Global Spitfire

Layer

SOAP

GGF Toronto 19.02.02

The Local Layer

Required from the local Site:


Any JDBC
-
enabled RDBMS backend.
(PostGreSQL,
Oracle, ..)


Any Servlet Container.
(Tomcat, Oracle Application Server,
WebSphere…)


SSL

Provides:


SOAP & WSDL interface


Role
-
based Authorization


Simple Persistency


Expiration based on a timestamp

GGF Toronto 19.02.02

The Global Layer

These are all only 'on the table' ; not worked out in detail, see
also Database BoF


Distributed Querying


Needs additional information on the given data, like
definition of common schemata and indices.


Caching/Replication mechanisms


Consistency


Expiration


Cleanup


Transactions

GGF Toronto 19.02.02

Local Layer Architecture


RDBMS

JDBC

Translator Servlet

SSL Auth

Security Servlet

Client API

Expiration Servlet

GGF Toronto
19.02.02

Security Mechanism

Servlet Container

SSLServletSocketFactory

TrustManager

Security Servlet

Does user specify role?

Map role to connection id

Authorization Module

HTTP + SSL

Request + client certificate

Yes

Role

Trusted CAs

Is certificate signed

by a trusted CA?

No

Has certificate

been revoked?

Revoked Certs

repository

Find default

No

Role repository

Role ok?

Connection

mappings

Translator Servlet

RDBMS

Request and connection ID

Connection

Pool

GGF Toronto 19.02.02

Local Layer Functionality


DB Administration


Create Database


Delete Database


Create Table


Drop Table


Role Administration


Create Role


Delete Role


Update Role


DB Information


Quotas


Memory, Disk space


User Info


Schemata


Core Functionality


Insert


Update


Delete


Select


Timestamps


Set Table Timestamp


Set Row Timestamp


Connections


Open Dedicated
Connection


Close Connection