Presentation - DuraSpace Wiki

gayheadtibburInternet and Web Development

Feb 5, 2013 (4 years and 2 months ago)

237 views

One Body, Many Heads

for Repository
-
Powered

Library Applications

Tom Cramer

Chief Technology Strategist

Stanford University Libraries


CNI * 13 December 2011

Repositories make strange bedfellows

University of Virginia, 2008

Virginia, Hull, Stanford & Fedora Commons
find common cause: leverage the power of
a repository for the full range of application
needs at our respective institutions.

.

Fundamental Assumption #1

No single system can provide the full range
of repository
-
based solutions for a given
institution’s needs,

…yet sustainable solutions require a

common repository infrastructure.

For instance…

-
Generally a
single PDF

-
Simple,
prescribed
workflow

-
Streamlined UI
for depositors,
reviewers &
readers


Digitization

Workflow

system

General Purpose
Institutional
Repository

Simple

Complex

-
Potentially
hundreds of files
type per object

-
Complex,
branching workflow

-
Sophisticated
operator (back
office) interfaces

-
Heterogeneous
file types

-
Simple to
complex objects

-
One
-

or two
-
step
workflow

-
General purpose
user interfaces

ETD

Deposit

System

Distinct Application Needs

More
than one dozen distinct repository
application needs across three
institutions.



Electronic theses & dissertations


Open access articles


Data
curation

application(s
)


General purpose institutional repository


Manuscript & archival collection delivery


Library materials accessioning tools


Digitization workflow system


And more...


Shared, Primitive Functions


Deposit


Manage


Edit Objects


Set Access


Search


Browse


Deliver


P
lus


Authentication


Authorization


Workflow

Hydra Philosophy
--

Technical


Tailored applications and workflows for
different content types, contexts and
user interactions


A common repository infrastructure


Flexible, atomistic data models


Modular, “Lego brick” services


Library of user interaction widgets


Easily skinned UI


One body, many heads

Technical Framework
-

Components


Fedora
provides a durable repository layer to
support object management and persistence


Solr
, provides fast access to indexed
information


Blacklight
, a Ruby on Rails plugin that sits
atop
solr

and provides faceted search &
tailored views on objects


Hydra Plugin
, a Ruby on Rails library that
provides create, update and delete actions
against Fedora objects

Blacklight for Repositories


Repository
-
agnostic, feature
-
rich, content
-
aware, turnkey access interface for repositories


Aggregate content from multiple repositories,
with links back to source systems


Vibrant, multi
-
institutional, open source
community on its own


Can be used independently, or as the first
component of, Hydra

CRUD in Repositories

Re
p
o
s
i
t
o
r
y
/
P
e
r
s
i
s
t
e
n
t

St
o
r
a
ge
C
r
e
a
t
e
/
Su
b
m
i
t
/
E
d
i
t
(CUD)
Se
a
r
c
h
/
V
iew
(
R)
CRUD in Repositories

Re
p
o
s
i
t
o
r
y
/
P
e
r
s
i
s
t
e
n
t

St
o
r
a
ge
C
r
e
a
t
e
/
Su
b
m
i
t
/
E
d
i
t
(CUD)
Se
a
r
c
h
/
V
iew
(
R)
Major Hydra Components

Fe
dora
So
l
r
So
l
r
i
z
e
r
B
l
a
c
k
l
i
gh
t
(
R)
hydr
a-head
Ra
i
l
s

P
l
u
gi
n
(CUD)
B
l
a
c
k
l
i
gh
t
(R
ead O
nl
y)
CRUD in Repositories

Fe
dora
So
l
r
So
l
r
i
z
e
r
B
l
a
c
k
l
i
gh
t
(
R)
hydr
a-head
Ra
i
l
s

P
l
u
gi
n
(CUD)
Act
ive
Fe
dora
OM
B
l
a
c
k
l
i
gh
t
(R
ead O
nl
y)
Fe
dora
So
l
r
So
l
r
i
z
e
r
B
l
a
c
k
l
i
gh
t
(
R)
hydr
a-head
Ra
i
l
s

P
l
u
gi
n
(CUD)
Act
ive
Fe
dora
OM
Hydr
a
Co
ntro
llers
Hydr
a
V
iews
B
l
a
c
k
l
i
gh
t
(R
ead O
nl
y)
Inside the Hydra
-
Head Plugin

The Full Architecture

MVC

Details

A Note on Ruby on Rails


Rapid

application development for web
applications:
“Convention over configuration”


10x productivity


Supportable:
MVC (Model
-
View
-
Controller) and
Rails framework make code well
-
structured,
predictable


Testable:
Rspec

and Cucumber give powerful,
automatable, testing tools


Learnable
: Stanford went from 1 to 8 Ruby savvy
developers in one year (no new hires)


1 week learning curve to basic proficiency



Fundamental Assumption #2

No single institution can resource the
development of a full range of solutions on
its own,

…yet each needs the flexibility to tailor

solutions to local demands and workflows.

Hydra Philosophy
--

Community


An open architecture, with many
contributors to a common core


Collaboratively built “solution bundles” that
can be adapted and modified to suit local
needs


A community of developers and adopters
extending and enhancing the core


“If you want to go fast, go alone. If you
want to go far, go together.”


One body, many heads

Community


Conceived & executed as a collaborative, open
source effort from the start


Initially joint
development project among
Stanford, University of Virginia,

and University
of
Hull


Close collaboration with
DuraSpace



Partnership with with
MediaShelf
, LLC


Complementary strengths and expertise

Actively Participating Institutions

0
2
4
6
8
10
12
OR09
OR10
OR11
Now
OR = Open Repositories Conference

Sustainability




No
animals

were
harmed


in the making of this
film
.

Job Postings

If you want to go fast…

…go alone.

…use Hydra?


Spring, 2011: Notre Dame deploys video
cataloging head in 6 weeks, from scratch



Ohloh.net

stats
(as of July 2011)


25 regular contributors in last 12 months


Top 2% of open source teams


Over $7.5 million in effort
<
snigger>


~9 person years of development

Hypatia

Development


8 week sprint

80/20


8 Weeks of Development

https://
github.com
/
projecthydra
/
hypatia
/graphs/impact

C
u
rre
n
t
l
y

-
D
u
ra
Sp
a
ce
-
H
u
l
l
-
Me
d
i
a
Sh
e
l
f
-
St
a
n
f
o
rd
-
V
i
rg
i
n
i
a
H
y
d
r
a

Ste
e
r
i
n
g

G
r
o
u
p
-
sma
l
l

co
o
rd
i
n
a
t
i
n
g

b
o
d
y
-
co
l
l
a
b
o
ra
t
i
ve

ro
a
d
ma
p
p
i
n
g

(t
e
ch

&
co
mmu
n
i
t
y)
-
re
so
u
rce

co
o
rd
i
n
a
t
i
o
n
-
g
o
ve
rn
a
n
ce

o
f

t
h
e

"t
e
ch

co
re
"
a
n
d

H
yd
ra

F
ra
me
w
o
rk
-
co
mmu
n
i
t
y
mt
ce
.

&
g
ro
w
t
h
H
y
d
r
a

Pa
r
tn
e
r
s
-
sh
a
p
e

a
n
d

d
i
re
ct

w
o
rk
-
co
mmi
ssi
o
n

"H
e
a
d
s"
-
f
u
n
ct
i
o
n
a
l

re
q
u
i
re
me
n
t
s

&
sp
e
cs
-
U
I

d
e
si
g
n

&
sp
e
c
-
D
o
cu
me
n
t
a
t
i
o
n
-
T
ra
i
n
i
n
g
-
D
a
t
a

&
co
n
t
e
n
t

mo
d
e
l
s
-
"U
se
r
g
ro
u
p
s"
Founders
-
D
u
ra
sp
a
ce
-
H
u
l
l
-
St
a
n
f
o
rd
-
U
V
a
Hydra Developers

-
d
e

n
e

t
e
ch

a
rch
i
t
e
ct
u
re
-
co
d
e

d
e
vl
e
o
p
me
n
t
-
i
n
t
e
g
ra
t
i
o
n

&
re
l
e
a
se
C
o
mmi
t
t
e
rs
C
o
n
t
ri
b
u
t
o
rs
T
e
ch
.

U
se
rs
Community

Model

Hydra
-
based
Applications at Stanford

ETD’s


Electronic Theses

& Dissertations

SALT


Self
-
Archiving
Legacy Toolkit

EEMs


Everyday
Electronic Materials

Argo


Repository Reporting
and Management

Hypatia



Archives &
Special Collections

Emerging Solution Bundles

Institutional Repositories

University of Hull

University of Virginia

Penn State University

Images

Northwestern University

(Digital Image Library)

Emerging Solution Bundles

Archives & Special Collections

Stanford University

University of Virginia

Rock & Roll Hall of Fame

Media

I
ndiana University

Northwestern University

Rock & Roll Hall of Fame

Etc.

Emerging Solution Bundles

Workflow Management
(Digitization, Preservation)

Stanford University

University of Illinois


Urbana
-
Champagne

Northwestern University

Exhibits

Notre Dame

Emerging Solution Bundles

ETDs

Stanford University

University of Virginia

Etc.

(Small) Data

everyone…

Philosophies


Building a framework, not an application
(variation is part of the plan)


Opinionated
software


Invest time & resources into collaborative
community (face time!)


Trainings & workshops


Openness, transparency (code, designs,
discussions)


Commit to contributing back to core


Design for re
-
use

Best Practices in Development


Agile, user
-
centric development process


Take a light touch when dealing with big topics


“Working software wins”


Distributed version control &
github


Unit testing


Cucumber testing


User testing


Continuous builds

Best Practices in Development (cont)


Weekly “stand up” meeting


Frequent deployment


JIRA, with user
-
centric
tickets


Documentation (ruby docs, wiki, etc.)

So What is Hydra?


Framework for generating Fedora front
-
end
applications w/ full CRUD
functionality


That follows design
pattern with common
componentry and platforms


Fedora, Ruby on Rails,
solr
,
ActiveFedora
,
Blacklight


That supports distinct UI’s, content types,
workflows, and
policies

So What is Hydra?


And a growing community of institutions and
developers committed to framework and
collaboration


Not grant
-
based


Distributed


Robust


Open

http://
projecthydra.org