scape - OPF wiki

hordeprobableBiotechnology

Oct 4, 2013 (4 years and 10 days ago)

86 views

SCAPE

Alexandra Nenadic,
David Withers

University of Manchester

Practical Tools for Digital Preservation: A Hack
-
a
-
thon

York, 28
th

September 2011

Taverna and
myExperiment

Tools for creating and sharing workflows

Taverna

SCAPE

What is a workflow?









Connecting a set of tools/services to create
automated and repeatable processing/analysis

SCAPE

Design and run workflows

Taverna

SCAPE

Taverna Features

-

Overview


Access to remote, distributed or local services and
resources


Enables service interoperability and integration


Automates data flow between services


Implicit iteration over data sets,
list handling and
control links to determine order of service invocation


Extensible


Large external developer base


Various third party
plugins

available


Data and provenance collection



SCAPE

Taverna

Workbench


Graphical desktop
tool


Drag
-
and
-
drop
services into diagram


Connect services, run,
reconnect, rerun


Integrates diverse set
of tools


SCAPE

Workflow Design

Available
services

Tree view of

the workflow
structure

Workflow
diagram

SCAPE

Taverna Workflows


Features in Detail


A set of (local and remote)
services

to analyze or manage
data


Data
-
links
connects services


i.e. output from service A is input to
service B and C


Describes the
desired dataflow
instead of process coordination


Nested

workflows are also services


Automatic iterations


Parallelization


Can customize
list handling
and
control links


Fault tolerance


Retry (with delay and back off)


Failover (alternate services)





SCAPE

Supported Services


SOAP/WSDL Web services


REST Web services


SoapLab

Web services


R statistical services


Inline
Beanshell

scripts


External tools and scripts (via
ssh

or
localhost
)


Spreadsheet
import


X
P
ath

and text manipulation services


SADI semantic Web services


Nested workflows (workflow within workflow)


BioMoby


BioMart




your

tool (write your own
Taverna

plugin
)


SCAPE

Workflow Results


Input data and
results per port

Progress report

Previous runs

SCAPE

Workflow Provenance


Information about a workflow run


What happened?


And when?


Lineage tracing


Which input produced which output


Intermediate data


Inputs and outputs for each workflow step


Useful for debugging


Saved in standard format (such as OPM)

SCAPE

Taverna is Domain
-
Independent


Bioinformatics


Biomedicine


Chemistry

Pharmacogenomics

Association study of
Nevirapine
-
induced skin
rash in Thai Population

HIV and TB research in
South Africa

Sleeping Sickness in
African Cattle

Systems Biology for
Crop research,
BioDiversity

SCAPE

Taverna is Domain
-
Independent


Astronomy


Data and text mining


Digital content preservation (IMPACT)


Social simulations


Astronomy &
HelioPhysics

Observing Systems
Simulation
Experiments

JPL, NASA

Library Document
Preservation

British Library

SCAPE

Share, discover and reuse workflows

SCAPE

myExperiment


http://www.myexperiment.org



Social networking for people to share workflows and
collaborate


Makes it easy for people to contribute to a pool of
workflows, build communities and form relationships


Enables people to share, describe, reuse and
repurpose workflows, reduce time
-
to
-
production,
share expertise and avoid reinvention

SCAPE

myExperiment

SCAPE

Workflow Sharing, Ownership and Attribution


myExperiment

can provide a central location for workflows
from one community/group


myExperiment

allows you to say


Who can look at your workflow


Who can download your workflow


Who can modify your workflow


Who can run your workflow


Workflow ownership and attribution


Users do not need to start from scratch


reuse or modify
existing workflows


Attribute/credit original author


SCAPE

Use
myExperiment

from Taverna

SCAPE

Training


Tutorials and Training


58+ tutorials to >900 people


>20 Universities, institutes
and networks


Major conferences


Summer schools


Developer and User Days


Annotation Jamborees


Undergraduate and
Postgraduate
Bioinformatics in > 30
Universities



SCAPE

Taverna and SCAPE


SCAPE preservation components/actions as services
in Taverna workflows


Use Taverna Workbench to create and test SCAPE
preservation workflows on local data


Then scale
-
up and run the workflows on a
parallelized platform using
Hadoop

MapReduce


Share Taverna SCAPE workflows on
myExperiment