Holding slide prior to starting

judgedrunkshipServers

Nov 17, 2013 (3 years and 6 months ago)

97 views

Holding slide prior to starting
show

COllaborative VIrtual TEams (COVITE) Project

-

P. Burnap, L. Joita, J.S. Pahwa
-


Overview


Project Concept


Collaborative Support Considerations


The Product Supplier Catalogue
Database (PSCD)


Security Management


User & Collaboration Management


Data Management


Conclusions and further work


AEC (Architecture/ Engineering/ Construction)
industry projects involve many individuals and
companies forming a consortium for the duration of a
project




Consortia members are geographically dispersed



Product/Service Manufacturers and Suppliers
databases (if existent!) are heterogeneous



Product/Service Manufacturers’ product information
is plentiful and the majority is unstructured and
unreachable



Solution?


A universal product supplier catalogue in
the form of a database


Project Concept

Collaboration occurs between:



Product Suppliers and Contractors for procurement of
supplies



Product Specification Designers for defining and
building industry standards to describe available
products



Members of the Consortium working on a particular
construction project which require information on the
products


Collaborative Support Consideration

The PSCD

The PSCD application


Grid
-
enabled data
management tool that provides the data structure for
storing information across a number of product
suppliers database.


The PSCD Functional Modules:


Security Management


User & Collaboration Management


Data Management

The PSCD

COVITE

Product Class Supplier Databases

Specification Designers

Users/User Groups

Supplier Databases managed by suppliers

Poll and connect
to relevant
databases at
runtime

Specification creation

and management keep

Suppliers up to date

active
plan
Cardiff University

PSCD

Application


PCD


Master Data

Security

Service

Cluster of GRID
computers

.NET Web
Services

.NET Web
Services

Master

Grid
Service

(MGS)

GRID
Services

Multiple
Database

Search
Service

(MDSS)

The PSCD


Software tools used:


Apache Tomcat web server, Axis, Ant


Microsoft IIS web server


GT3.0.2 core


Java CoG 1.1a


Servlet, ASP, JSP, VBScript, C#

Security Management


Security considerations:


Authentication


Authorization


Confidentiality


Integrity



Security Management defines a security
framework for the PSCD application using the
Globus Security Infrastructure (GSI).

Security Architecture

1.
First, a user has to have a valid proxy
certificate. The user submits his proxy
certificate to the Tomcat
Authentication Server (AS) via a web
interface using JSP
-

servlet interface.

2.
Tomcat AS authenticates the
certificate and obtains the local user
name for the PSCD application from
the grid
-
mapfile.

3.
Tomcat AS passes the local user
name to the IIS server that runs the
PSCD system (which is a .NET web
application environment). IIS then
matches the username to its local DB
and creates a session for that user.

4.
User preferences are applied to the
‘index’ page of the PSCD system and
the user is presented with the home
page of the application.

Client Side Web Browser

Firewall

Tomcat

Authentication

Server

IIS

Application

Server

3

PSCD Application

Submit user
proxy
certificate

2

4

1

The PSCD Authentication Architecture using a valid proxy certificate


Security Architecture

1.
First, a user has to have a valid proxy
certificate on the MyProxy Server machine.

2.
The user submits his username/password
pair credentials to the Tomcat
Authentication Server (AS) via a web
interface using JSP
-

servlet interface.

3.
Tomcat AS uses the username/password
pair to authenticate and authorize the user
against his proxy certificate from MyProxy
server

4.
Tomcat AS authenticates the proxy
certificate upload from MyProxy server and
obtains the local username for the PSCD
application from the Grid
-
map file.

5.
Tomcat AS passes the local user name to
the IIS server that runs the PSCD system
(which is a .NET web application
environment). IIS then matches the user
name to its local DB and creates a session
for that user.

6.
User preferences are applied to the ‘index’
page of the PSCD system and the user is
presented with the home page of the
application.

The PSCD Authentication Architecture using username/password credentials

Client Side Web Browser

Submit
username/

password

1

Firewall

Tomcat
Authentication
Server

MyProxy
Server

IIS

Application
Server



2

3

4

5


6

PSCD Application

Security Management

Demonstration:

http://131.251.42.37:8080/CovitePortal

Communication


Human
-
Computer Interaction



Product Class Creation


Product Class Subscription


Product Searching


Procurement


Project Design


Project Virtual Meetings

Current Work

Sub Product Class
Specification

Specification Group

Specification

Sub
-
Specification

Group

Table Spec

List Specification

Product Class
Specification

Specification


Value

Table Specification

List of Column
Specifications
where each
column
specification has
list of values
relative to other
column
specification
values.



List Specification


Value


Value





Product Class

The Product Class and its various
specification types

Demonstration

Product Class Versioning


New products emerge


Existing products evolve and are assigned
more attributes


New versions created and assigned


Product suppliers notified


New classes downloaded



Product Class Subscription

Table

Table





Table



Table






Xml

Parse

Service

<xml>










</xml>

Product Catalogue
Database

Transport Layer

Transport Layer

Product Supplier
Database

Demonstration

Resulting File

Sub Product
Class
Specification

Specification Group
ID

Specification

Sub
-
Specification

Group ID

Table Spec

List Specification

Product Class
Specification

Specification


Value

Table Specification

List of Column
Specifications
where each
column
specification has
list of values
relative to other
column
specification
values.



List Specification


Value


Value





Product Class

The Product Class and its various
specification types

Recursive Algorithm

doSearch (classID)
//
Returns class details, spec objects, tables, lists and groups

dataSet.Merge (classDetails)


Foreach (groupID in classDetails)
// Same method used for class retrieval

{



findChildren (groupID)
// Returns all details of spec groups inc. children IDs


dataSet.Merge (children)

}


getChildren (childrenIDs)
// RECURSIVE METHOD


Do

{


getChildren (childrenIDs)
// Searches for spec groups by parent’s ID


dataSet.Merge(children)

}

While (children != null)
// While there are still children under the parent IDs


dataSet.WriteXml("C:/Inetpub/sql2000/classDetails.xml");


XML Parsing


Product supplier receives XML dataset file


.NET System.Xml namespace used to parse
the file and pulls out each element of the
product class by locating nodes and values


Traverses the document sequentially without
caching data in memory


.NET XmlReader classes use a ‘Pull’ model
as opposed to SAX’s ‘Push’


New product class details populated to PSCD


Data Management


Information held by large number of data sources in a
Wide Area Network.


Access to such information held by large number of
organisations.


Information processing with aid of independent
mechanisms, and


Sharing of information with those who need it.




The Approach



Does a data source really need to be “Grid Enabled”
to provide data access support to the Grid
Environment?


-

may be not.



Instead, why not just Grid enable the data access
process?


-

grid enabled data sources and grid enabled data

access are two different things.

The MDSS


Multiple Database Search Service.


A database centric Grid service for accessing
and processing information from large
number of data sources.


Searches supplier data sources based on a
search criteria.


Built using Globus Toolkit 3.0.2 (Core).



The Search


What to search?


-

a search criteria submitted by the user.


Where to search?


-

data sources that match user’s request.


What are the available resources to propagate the
search?


-

the available Database Search Services in

the MDSS

The Objective


Searching autonomously managed data sources
external to the Grid Environment.


Processing, analysing and sharing information in real
time.


Collaborative working.


Collation of searched datasets.


Optimisation of data access operations.


Web Services support throughout.


Use of single proxy.




The Architecture

Two major components:


Master Grid Service: identify resources available,
register new resources, job allocation, data parsing and
data aggregation.



Database Search Service: data search, data
aggregation, data parsing,

collaborative working etc.




MDSS Architecture

WS Layer

WS Layer

WS Layer

WS Layer

WS Layer

WS Layer

Grid DB Search Service

Grid DB Search Service

Grid DB Search Service

Apache Axis Soap Server

<<soap messages >>

<<soap messages >>

<<soap messages >>


Master Grid Service

* Metadata Query *Job Allocation Grid Service * Collation of datasets


Single user

VO

Active Plan

(Search Criteria Specified here)

Metadata

Product Classes

Grid Engine

(based on the OGSA Model)

Supplier databases

Design Features


Ability to register new data sources.


Ability to enhance MDSS capability by adding more
machines in the Grid cluster.


Supplier databases free of Grid Complexities.



Ability to identify data sources to be searched in real
time.


Distribute search equally among available GSHs.


New instance of Database Search Service for each
operation.


GSH

Document

Parser



Supplier

Document

Parser




Xml

Converter



Database

Search

Service


Job

Execution


Grid

Service

Manager


Data

Aggregation



Master

Grid

Service


OGSA Container

Master Grid Service Modules

Search Criteria

Where to Search?

<NewDataSet>


<Supplier>


<IDSupplier>1</IDSupplier>


<SupplierWS>http://131.251.42.40/test/categories.asmx</SupplierWS>


</Supplier>




</NewDataSet>


Available GSHs


<NewDataSet>


<GridServiceHandle>


<IDGsh>1</IDGsh>


<GSH>http://131.251.42.40:8080/ogsa/services/DatabaseSearchService</GSH>


</GridServiceHandle>


<GridServiceHandle>


<IDGsh>2</IDGsh>


<GSH>http://131.251.47.110:18080/ogsa/services/DatabaseSearchService</GSH>


</GridServiceHandle>




</NewDataSet>


Demonstration


Submit search Job to Master Grid Service.


Master Grid Service distributes the search across 4
Database Search Services registered with the MDSS.


The Database Search Services collaborate to retrieve
information from 150 data sources.



http://131.251.42.40/test/GridServiceTest.aspx


Issues


Identification of Grid resources capability.


Complicated procedure of setting up grid services.


Unpredictable results when creating port types.


Scalability


large number of simultaneous operations.


Lack of documentation.





Conclusions & Further Work


Web Service Security Framework.


Peer Review System.


Multiple instances of Master Grid Services.


Data Streaming.


Data Caching.


Memory Resident databases.


Integration of MDSS and the PCD System with Security
Service.