RepoMMan: using Web Services and BPEL to facilitate workflow interaction with a digital repository

RepoMMan: using Web Services and BPEL to facilitate

workflow interaction with a digital repository

Richard Green

Overview

•
RepoMMan Project overview

•
User and University needs

•
A solution: BPEL and Web Services

•
Example workflows: deposit and metadata

•
Problems? What problems?

•
Future work

JISC Workflows activity and implementation meeting


Aston Business School

Tuesday 13
th

February 2007

RepoMMan project


Repository, Metadata and Management project



JISC
-
funded for two years to end May 2007...



...to develop a BPEL
-
based, standards
-
compliant, Web
Services based, workflow tool for Fedora



Closely aligned with the University’s commitment to
deploy an institutional repository

RepoMMan


Two strands:



research
–

user needs, documentation etc (BPEL, surveys,


Beginner’s guide etc)



technical
–

development



Surface tool in University portal and/or Sakai C&LE

Hull’s vision for a DR


Hull’s vision for an Institutional Repository is an
extremely broad one



Conventional view was exposure of completed objects



Hull’s view encompasses storage, access, management
and preservation of a wide range of file types from
concept to completion

User needs


Survey and interviews revealed very wide range of file
types potentially to deal with


Users need


Storage (backed up)


Access (from anywhere)


Management (sharing, locking, versioning etc)


Some want preservation


Tool must fit users’ expectations and environment


Assist/improve doing what they already do


University needs


Flexible


wide range of content, some public
-

some private, appropriate
UIs


Standards
-
based


Web Services, standards
–

BPEL, JSR168 etc


Scalable


initially 2500 potential depositors


Open source software


Effective search & discovery


need good metadata

The repository grand plan

Summary


The repository will be a working environment as
well as a showcase


Aim is to provide storage, access, management
and potential preservation painlessly


Development based on user needs


Standards compliant, Web Service approach

Solution


Toolset to manage repository workflows for
users


BPEL
-

Business Process Execution Language


(Active Endpoints Open Source version)


Web Services


Fedora repository software


SOAP, REST


scales to millions of flexible objects (NSDL = 6m+)


BUT


Fedora is not an ‘out
-
of
-
the
-
box’ solution


Provides the repository functionality for YOU to
use flexibly

Basic depositor workflows


Put object into repository


including some sort of structuring


Get object from repository


Delete


including at version level


Add metadata


Publish


= promote to public space

The RepoMMan deposit tool

‘Putting’ and ‘getting’ a file


Left hand side of screen gives browse capability
on user’s hard drive


Right hand side of screen shows the user’s
private repository space like a directory structure


(‘Folders’ are in fact repository collections)


Folders can be expanded and new folders
created


Objects can be further expanded to show
versions

Three
-
tier stack

Three
-
tier stack


Model View Controller layer providing user
interface


BPEL orchestrating Web Services (Fedora and
other) to move files and objects around


Fedora drawing on ID Management System and
University Storage Area Network

UML diagram for simple deposit

Three
-
tier stack: simple deposit

‘Putting’ a file

Part of the BPEL process diagram

(Active Endpoints visualisation software)


-

switch depending on whether


object already exists


-

the left hand side branch creates


a new object


-

the right hand side modifies an


existing one


-

each of the globes with a ‘swirl’


round it is a Web Service call

BPEL and web services


BPEL can draw on any available Web Service


Access to an expanding range of data made
available through Web Services at Hull (and, of
course, elsewhere)



Which brings us to metadata!!!

Making public


Good metadata at the heart of effective search and
discovery



When a user wants to ‘make an object public’ we will
automatically add metadata



Potentially complex: BPEL to orchestrate



Has object already got metadata?


Derive metadata using tool(s)


Extract appropriate parts


Combine with contextual metadata


Allow user to edit


Create datastream(s) (DC, Rich metadata (UVa), preservation
metadata, etc)


Add to digital object


Metadata from web services


Much contextual metadata about the ‘author’
can be derived from University systems’ Web
Services or context



Technical metadata can be derived by using
tools such as JHOVE (as a Web Service)



Descriptive metadata about the content is the
Holy Grail?


Data Fountains (UC Riverside & NSDL)




Metadata from context & web svcs


Contextual
metadata from
environment
(LDAP, Portal,
Sakai)



Technical metadata
(JHOVE etc)



Additional
descriptive
metadata from
content (Data
Fountains etc)



Allow user to edit /
tweak


Acknowledgements to the Arrow consortium

for the design idea

Problems? What problems?


Not a smooth journey!


Lessons: Don’t be a pioneer? Puppies don’t come free?


Active BPEL was/is fine


Fedora Web Services (v2.0, v2.1, v2.1.1) an issue


Web Services were rpc/encoded


BPEL and AXIS couldn’t always successfully validate responses


Fedora Web Services (v2.2) OK!


Web Services now document/literal

Next stages


Complete the RepoMMan deposit tool (!!!)


Internal testing


Trialling with users


Small
-
scale trials


Complete the RepoMMan metadata work


Post
-
project, develop further and deploy


Develop workflow tools for the admin side of the ‘make
public’ process, records management, preservation...


In time switch to DROID (for MIME type)? Shibboleth?


Project website and contacts

RepoMMan: http://www.hull.ac.uk/esig/repomman


Contacts:


r.green@hull.ac.uk (Manager)


c.awre@hull.ac.uk (Integration architect)


i.dolphin@hull.ac.uk (Director)


s.lamb@hull.ac.uk (Software developer)