eSciDoc Persistence Layer

batterycopperInternet and Web Development

Nov 12, 2013 (3 years and 7 months ago)

117 views

eSciDoc
Persistence

Layer

Architecture

Page

Overview

1.
General eSciDoc architecture

2.
Business Layer Excursion

3.
Persistence Layer

4.
Transformation to the module/dependency structure

2

Page

Overview

3

Page

1. General eSciDoc
architecture

Web
Application

Spring Framework

Persistence

Layer

Business
Logic

Layer

Communication

Layer

Transport

Layer

CXF

Security Interceptor

Service
Spec
.

REST
Endpoint

Spec
.

JMS

Transport

JAXB

REST
Endpoint

Impl
.

SOAP
Endpoint

Spec
.

SOAP
Endpoint

Impl
.

HTTP

Transport

FastInfoSet

Interceptor

Service
Impl
.


XOP Interceptor


GZIP Interceptor

Repository
Spec
.

FEDORA

Repository
Impl
.

Transaction Interceptor

JSON


Hibernate

FEDORA

RMDB

implements

depends

Transfer

Objects

Domain

Objects

Persistent

Objects

map

map

4

RMDB

Repository
Impl
.

Page

1.1 Focus

Web
Application

Spring Framework

Persistence

Layer

Business
Logic

Layer

Communication

Layer

Service
Spec
.

Endpoint

Spec
.

Endpoint

Impl
.

Service
Impl
.

Repository
Spec
.

Repository
Impl
.

Data

Maintenance

System

implements

depends

Transfer

Objects

Domain

Objects

Persistent

Objects

CL
-
Mapping

PL
-
Mapping

5

Page

1.2 Mapping

CL
-

& PL
-
Mappings are part of each layer


Only the layer knows, which peaces of the business data belongs to the objects of the
layer


Example: Hibernate


The Hibernate Layer defines the data structure to be used to persist the data (tables, indexes, etc.)


Hibernate Model A: Use one table to store Items and Container


Hibernate Model B: Use two different tables to store Items in one table and Containers in the other table


Example: Fedora


Fedora uses FOXML to persist the data


Example: Communication Layer (CXF)


Defines Transport
-
Objects which are
marshalled

to XML, JSON etc.


Because of the different data structures each concrete layer instance implements, it is
the layers responsibility, to map the business domain objects to its own data model


The business logic is completely independent of anything that happens outside of its
own borders

6

Page

2. Business Layer

A little excursion for later reference

Business Logic Layer

Service
Spec.

Service Impl.

updateItem()

updateMdRecords()

updateMdRecord()

updateContainer()

Domain
Objects

Item

Container

MdRecords

MdRecord



Persist
ence
Spec.

Uses Domain Object

Possible Interface/Call

Internal call to sub
-
method

7

Page

2. Implementation structure

Structured, centralized and reusable code


Sub
-
resource methods reused by the service
-
methods handling the main resources


Avoid conceptual flaws on implementation of new features, bug fixes and so on
(protect the developer by design)

Service methods implement the eSciDoc logic like


Update a resource:


What happens, if the resource from the client equals the currently persisted resource
and what happens if not?

Service methods use the Domain Objects as data holders, which only
implement logic, that belongs to their semantic like


getObjid
() : extracts the
objid

from the
xlink:href


getVersionNumber
() : extracts the version number from
getObjid
()

Thanks to this concept, sub
-
resource methods can be reused and offered as
services to the communication layer as well


8

Page

2. Flow example

9

updateItem
(arg0)

updateMdRecords
(arg0, arg1)

updateMdRecord
(arg0, arg1)

updateMdRecords
(
mdRecords
, null)

updateItem
(item, null)

updateMdRecord
(
mdRecord
, null)

arg1 == null ?

arg1 =
retrieveMdRecords
()

arg1 =
retrieveItem
()

arg0.hasMdRecords() & arg1.hasMdRecords()

updateMdRecord
(arg0.getMdRecord(
name
),


arg1.getMdRecord(
name
))

arg1 == null ? arg1 =
retrieveMdRecord
()

arg0 != arg1 ?
persistMdRecord
()

CL

PL

updateMdRecords
(arg0.getMdRecords(),


arg1.getMdRecords())

arg0.hasMdRecord(
name
) & arg1.hasMdRecord(
name
)

Page

2. Loss
of

information


Providing central methods for reusability will cause a loss of information


eSciDoc interfaces are structured to support different resource types per interface


The centrally provided methods however do not differ between these resources, when
handling sub
-
resources


The persistence layer may still need the information about the type of the resource


If the Hibernate Model B is being used


Or if the data is stored in different ways into the FOXML, depending on the
resource type


Therefore we have to support the
ResourceType

for these methods, not matter if we
need the type or not


Support future expandability and plug
-
in ability for the persistence layer


Signature example:
updateMdRecord
(
MdRecord
,
MdRecord
,
ResourceType
)


Support an extra package/module for the static sub
-
resource handlers for better
understanding of the structure


10

Page

3. Persistence Layer

Goals


Current

goal

for

SCAPE:
Use

Hadoop

as

a
persistence

layer


Plug
-
in ability


In theory eSciDoc could get released without an already implemented persistence
layer


How the customer can plug
-
in an own implementation of the persistence layer is
not part of this presentation
(
How to do this in Tomcat,
JBoss

etc.)


Black box for the business layer


Exception

handling


Mapping
of

the

business

domain

objects

to

its

own

objects

11

Page

3.
Persistence

Layer
as

a Black Box


Exception handling


Exceptions

thrown

in
the

persistence

layer

cannot

be

handled

in
detail

by

the

business

layer


The
business

layer

does

not
know
,
what

kind

of

persistence

layer

is

being

used


Plug
-
in
support

offers

customers

to

implement

their

own

persistence

layer
,
we

do not
even

know

anything

about

their

implementations


The
business

layer

may

define

an
abstract

PersistenceException
,
which

will
be

the

representative

of

all
exceptions

thrown

in
the

persistence

layer


If

we

have

some

code

in
the

current

business

layer
,
handling

such
persistence

layer

exceptions
,
we

have

to

analyze

this

code

and

find a
better

solution

for

the

idea

of

this

code


ResourceNotFoundException


Return null instead of throwing an exception in the persistence layer and throw a
ResourceNotFoundException

in the business layer because it is the business
logic decision and design to throw an exception, if a resource was not found

12

Page

Persistence

Layer

Business Layer

Communication Layer

3.
Exception

handling



Flow
example

(
ResourceNotFoundException
)

13

retrieveItem
()

Retrieve

Item

Retrieve

Item

Retrieve

Item

Load

data

Throw

404
exception

return

exception

XML

Return Item

return

resource

XML

Resource

found

Delegate

Persistence

Exception

Error
-
like

exception

IO)

404
exception

OR
empty

result

OR
anything

alike

Return null

throw

PersistenceException

Return
resource

Page

3.
Exception

handling

-

Outcome


Avoid conceptual flaws on implementation


Non
persistence

layer

exceptions

should

not
be

available

for

the

persistence

layer


Adjust

eSciDoc
module

hierarchy


Adjust

the

persistence

layer

interfaces

to

describe

the

expected

behavior

of

the

business

layer


Implement

the

mapping

between

business

domain

objects

and

persistence

layer

objects

as

already

described

14

Page

3. Plug
-
in
ability

Now

we

can

add

any

kind

of

persistence

layer

implementation

to

the

business

layer

by

making

them

configurable

in Spring

15

Business
Logic

Layer

Service
Spec
.

Service
Impl
.

implements

depends

Domain

Objects

Hibernate

Model A

Repository
Spec
.

Repository
Impl
.

Data

Maintenance

System

Persistent

Objects

PL
-
Mapping

Hibernate

Model B

Repository
Spec
.

Repository
Impl
.

Data

Maintenance

System

Persistent

Objects

PL
-
Mapping

Fedora

Repository
Spec
.

Repository
Impl
.

Data

Maintenance

System

Persistent

Objects

PL
-
Mapping



Page

3. Plug
-
in
ability

But
wait



There

is

something

very

wrong

with

this

architecture
!

16

Page

3. Plug
-
in
ability

Each

Persistence

layer

defines

its

own

specification
.

17

Business
Logic

Layer

Service
Spec
.

Service
Impl
.

implements

depends

Domain

Objects

Hibernate

Model A

Repository
Spec
.

Repository
Impl
.

Data

Maintenance

System

Persistent

Objects

PL
-
Mapping

Hibernate

Model B

Repository
Spec
.

Repository
Impl
.

Data

Maintenance

System

Persistent

Objects

PL
-
Mapping

Fedora

Repository
Spec
.

Repository
Impl
.

Data

Maintenance

System

Persistent

Objects

PL
-
Mapping



Page

3.
Persistence

Specification


If the specification is part of the persistence layer, the business logic has to implement
a switch

in order to select and work with the current persistence layer


The
business

logic

has

to

implement

special

code
,
which

deals

with

each

persistence

layer

specification


The
business

logic

layer

depends

on

the

persistence

layer


The
business

logic

layer

cannot

specifiy

the

expected

behavior

of

the

persistence

layer

(
for

example
:
the

exception

handling
)


The
persistence

layer

specification

expects

the

persistent
objects

in
its

signatures

and

because

of

that

the

business

logic

has

to

perform

the

mapping

from

its

domain

objects

to

the

persistent
objects
,
no

matter
what

kind

of

persistence

layer

we

are

using


The
persistence

layer

is

no

real
black

box


A
lot

of

ugly

code

will
be

written



All
this

conflicts

with

the

idea

to

support

the

plug
-
in
ability

and

to

achieve

an
easy
extension

to

use

Hadoop

for

example

18

Page

3.
Corrected

architecture

Web
Application

Spring Framework

Persistence

Layer

Business
Logic

Layer

Communication

Layer

Service
Spec
.

Endpoint

Spec
.

Endpoint

Impl
.

Service
Impl
.

Repository
Impl
.

Data

Maintenance

System

implements

depends

Transfer

Objects

Domain

Objects

Persistent

Objects

CL
-
Mapping

PL
-
Mapping

19

Repository
Spec
.

Page

3.
Outcome


The
business

logic

layer

defines

the

persistence

specification


Uses

its

domain

objects

in
the

signatures


Predefines

the

expected

behavior

(
Exception

handling

etc.)


Uses

the

PersistenceException

in
the

signatures


Each

persistence

implementation

implements

this

one

specification

and

performs

its

own

mapping

from

domain

objects

to

its

own

persistent
objects


The
persistence

layer

becomes

a real
black

box


The
business

logic

layer

can

easily

work

with

any

persistence

layer

implementation

without

the

need

to

implement

special

code

for

any

persistence

layer

implementation


The
persistence

layer

is

pluggable

like

the

communication

layer


20

Page

4. eSciDoc Modules

> Communication Layer

21

context
-
rest
-
impl

context
-
rest
-
spec

dataobjects
-
impl

dataobjects
-
spec

jax
-
rs

framework

om

policies

aspects

common

aa

utilities

spring
aspects

FastInfoSet

geronimo
-
servlet

3.0 spec

woodstox
-
core
-
asl

CXF

frontend
-
jaxrs

Transports (HTTP,
Jetty
, JMS)

Business
logic

spec
.

Communication Layer

Business Logic Layer

BL
dataobjects

(
domain

objects
)

Mapping

Page

4. eSciDoc Modules

> Communication Layer

22

context
-
rest
-
impl

context
-
rest
-
spec

dataobjects
-
impl

dataobjects
-
spec

jax
-
rs

framework

om

policies

aspects

common

aa

utilities

spring
aspects

FastInfoSet

geronimo
-
servlet

3.0 spec

woodstox
-
core
-
asl

CXF

frontend
-
jaxrs

Transports (HTTP,
Jetty
, JMS)

Before

we

continue
,
we

have

to

clean
this

up
.

Mapping

Page

4. eSciDoc Modules

> Communication Layer

23

context
-
rest
-
impl

context
-
rest
-
spec

dataobjects
-
impl

dataobjects
-
spec

jax
-
rs

framework

om

policies

aspects

common

aa

utilities

spring
aspects

FastInfoSet

geronimo
-
servlet

3.0 spec

woodstox
-
core
-
asl

CXF

frontend
-
jaxrs

Transports (HTTP,
Jetty
, JMS)

?


Dependencies

are

resolved

transitively.



aspects
“ & „
policies

should

be

included

in
each

root

module

only


Business
logic

modules

cannot

be

used

in
the

communication

layer

and

vise

versa
.


Only

the

business

logic

specification

should

be

used

in
module


context
-
rest
-
impl



Mapping

Page

4. eSciDoc Modules

> Communication Layer (
Discussion
)


dataobjects
-
impl

depends

on
aa


Reason

unknown


Dependencies

utilities

and

common


Reusability


Not
allowed

between

layers


Create a
utility

module

outside
of

the

layers


Dependency

to

om


Reason

unknown

24

Page

4. eSciDoc Modules

> Communication Layer (
Cleanup
)

25

context
-
rest
-
impl

context
-
rest
-
spec

dataobjects
-
impl

dataobjects
-
spec

jax
-
rs

framework

om

policies

aspects

common

aa

utilities

spring
aspects

FastInfoSet

geronimo
-
servlet

3.0 spec

woodstox
-
core
-
asl

CXF

frontend
-
jaxrs

Transports (HTTP,
Jetty
, JMS)

?

Business
logic

spec
.

Communication Layer

Business Logic Layer

Mapping

BL
dataobjects

(
domain

objects
)

Page

4. eSciDoc Modules

> Communication Layer (Final)

26

context
-
rest
-
impl

context
-
rest
-
spec

dataobjects
-
impl

dataobjects
-
spec

jax
-
rs

framework

policies

aspects

spring
aspects

FastInfoSet

geronimo
-
servlet

3.0 spec

woodstox
-
core
-
asl

CXF

frontend
-
jaxrs

Transports (HTTP,
Jetty
, JMS)

Business
logic

spec
.

Communication Layer

Business Logic Layer

Mapping

BL
dataobjects

(
domain

objects
)

Page

dataobjects

4. eSciDoc Modules

> Communication Layer (Final)

27

context
-
rest
-
impl

context
-
rest
-
spec

dataobjects
-
impl

dataobjects
-
spec

jax
-
rs

framework

policies

aspects

spring
aspects

FastInfoSet

geronimo
-
servlet

3.0 spec

woodstox
-
core
-
asl

CXF

frontend
-
jaxrs

Transports (HTTP,
Jetty
, JMS)

Business
logic

spec
.

See
architecture

on
page

19

Use

woodstox

as

the

parser

impl
.
for

JAXB

General
servlet

spec
.

(Also
works

for

JBoss
)

?

No

need

to

differ

between

spec
. &
impl
.
for

objects

Communication Layer

Business Logic Layer

call

implement

Mapping

BL
dataobjects

(
domain

objects
)

international standard that
specifies a binary encoding
format for the XML
Information Set (for JMS?)

Page

4. eSciDoc Modules

> Business
logic

layer

28

Business
logic

spec
.

Business
logic

Impl
.

common

policies

aspects

Persistence

layer

spec
.

aa

Sub
-
resource

handler

See
page

19

See
page

7
-

14

Optional

(
Should

be

replaced

with

an HTTP
connection

or

using

CXF in
terms

of

clustering

abilities

later
)

Other
dependencies

(
no

eSciDoc
modules
)

BL
dataobjects

(
domain

objects
)

implement

call

Page

4. eSciDoc Modules

> Business
logic

layer

(
Discussion
)


dataobjects
-
spec

is

required

for

SRU
classes

for

example

but
should

not
be

part

of

this

layer

(
RequestType
,
ResponseType
, etc.)


Create
dataobjects

(
domain

objects
)

module

for

business

layer
,
where

the

domain

objects

are

being

implemented

and

external

objects

are

generated

from

XSD
schema

files


Code
about

mapping

between

String &
DateTime

should

not
occur

in
this

layer

(
DateTimeJaxbConverter

from

the

Communication Layer)


Put

such
utility

classes

in an
utility

module
,
which

can

be

reused

on
any

layer

and

is

not
part

of

any

layer


Rename

DateTimeJaxbConverter

to

DateTimeConverter

(
for

example
)

29

Page

4. eSciDoc Modules

>
Persistence

layer

30

Persistence

layer

Impl
.

policies

aspects

Persistence

layer

spec
.

Other
dependencies

(
no

eSciDoc
modules
)

PL
dataobjects

BL
dataobjects

(
domain

objects
)

Mapping

implement

Business Logic Layer

Persistent Layer

Page

4. eSciDoc Modules

>
Persistence

layer

(
Discussion
)


Nothing

31

Page

4. Project Structure & Dependencies

32

eSciDoc BL Project

eSciDoc CL CXF Project

eSciDoc PL Fedora Project

Domain Objects

BL
Spec

PL
Spec

BL
Impl
.

Utilities

Page

4. Project Structure & Dependencies


Advantages


The borders of the layers are clear


Protects the developer to spuriously add wrong dependencies


Maybe add some kind of Maven plug
-
in, which evaluates the allowed eSciDoc
module dependencies


If a new persistence or communication layer is being implemented, a new project
for this new layer can be created


Avoids adding a bunch of new modules to the current project for the new layer
implementation(s)


The Utilities module has to be completely independent from any layer


If that is not possible, because you may need objects from one layer, it has to be
part of this layer and cannot be reused in another layer!


33

Page

Addendum A
-

Layers


Why do the CL, BL and PL layers looks so differently?


Let us look at this using another perspective and some examples

34

BL

BL Spec

PL Spec

CL CXF
REST

CL Axis
SOAP

GUI

PL Fedora

PL
Hibernate

PL
Hadoop

Outgoing

Incoming

call

implement

“Plug
-
in”

Request

Request

Page

Addendum B


BL Project Structure


Protect the developer by design


Reusability of some modules for internal implementations of CL and PL

35

<POM>

BL

<POM>

Tools

<POM>

Core

<POM>

API

<JAR>

JAX
-
RS Framework

<JAR>

Utilities

<JAR>

AA

<JAR>

OM

<JAR>

Domain Objects

<POM>

PL API

<POM>

BL API

<JAR>

AA PL API

<JAR>

OM PL API

<JAR>

AA BL API

<JAR>

OM BL API







Page

Addendum B


BL Project Structure

Artifact Group
-
IDs

36

<POM>

BL

<POM>

Tools

<POM>

Core

<POM>

API

<JAR>

Domain Objects

<POM>

PL API

<POM>

BL API

org.escidoc.bll.tools

org.escidoc.bll.core

org.escidoc.bll.api.domain

org.escidoc.bll.api.persistence

org.escidoc.bll.api.business

Page

Addendum B


BL Project Structure


Tools


Offer our JAX
-
RS Framework as a “service” to be reused by the CL & PL
implementations as we require it in our current implementations


Like the
RestEASY

service from
JBoss


Offer some general and
independent

Utilities


API


Domain Objects


Required by the BL API and PL API and therefore have to be available to the
public as well


PL API


Separated into each eSciDoc module (
aa
,
om
,
oum

etc.)


BL API


Separated into each eSciDoc module (
aa
,
om
,
oum

etc.)


Core


Separated into each eSciDoc module (
aa
,
om
,
oum

etc.)

37

Page

Addendum B


BL Project Structure


Pros of this structure


You can see the borders within the BL and therefore know, which modules
can

depend on each other and where dependencies are
forbidden


No dependencies are allowed
across

Tools
,

Core
and

API


Tools

and
API

are the only module groups, which can be referenced by other
projects/layers (such as PL, CL and the BL itself)


No layer (PL or CL) is allowed to depend on modules of the
Core

module group


Implement a Maven plug
-
in, which evaluates the dependencies and causes the
build process to fail, if a dependency to the
Core

module group exists


Simply do not allow dependencies to
org.escidoc.bll.core
*


Use this plug
-
in in CL & PL projects and in
API

&
Tools

module group too

38

Page

Addendum B


BL Project Structure

Reusability


Our internal implementations of CL and PL use the JAX
-
RS Framework module as
their CXF implementation


Avoid redefinition of the same module in both layers


BL may use this framework too

39

<POM>

BL

<POM>

Tools

<JAR>

JAX
-
RS Framework

<JAR>

Utilities

PL

CL

Page

Addendum C


Example for Context

40

JAX
-
RS Server

context
-
rest
-
spec

context
-
rest
-
impl

context
-
spec

context
-
impl

context
-
pl
-
api

context
-
pl
-
impl

FedoraRestClient

FedoraRestClientImpl

FedoraRESTEndpoint

JAX
-
RS Client

CL

BL

BL

PL

TODO

External (not a module)

Page

Addendum C


Example for AA

41

JAX
-
RS Server

aa
-
rest
-
spec

aa
-
rest
-
impl

aa
-
spec

aa
-
impl

aa
-
pl
-
api

aa
-
pl
-
impl

Hibernate

Hibernate
Impl

CL

BL

BL

PL

TODO

External (not a module)

Thank

you

!

These documents are intended for presentation purposes only.

Copyright lies with FIZ Karlsruhe.

Any distribution or use of these documents or part thereof is

subject to FIZ Karlsruhe's express approval.


© FIZ Karlsruhe 2011