I2Home - My Opera

arghtalentData Management

Jan 31, 2013 (4 years and 7 months ago)

158 views

1

Shards

Object Database for Java5+


Vaclav Slovacek


vvessan@gmail.com

my.opera.com/wessan

2

Similar projects


Caché


cross
-
platform


DB4O


Java a .net


Project Darkstar


Java


MMORPG


NeoDatis


ObjectDB


OzoneDB

3

Shards Evolution


partial serialization


iteratively store changes in huge object networks


similar to Darkstar


ODB for Java a .net


using existing API (for example JDO)



similar to DB
4
O


Java byte
-
code manipulation


new API

4

#1

Ease of Use

5

#2

Reliability

6

Features


minimize amount of work for developer


100
% pure Java


encouraging usage of recommended OO techniques


very small API


no additional query language


small size (under
2
MB)



no collision with serialization, JPA, JDO, etc.

7

Querying

Lazy

Loading

Ease of Use ?

Transaction

Processing

Journaling

Indexing

Garbage

Collection

Bytecode

Analysis

Caching

Security

Class

Versioning

Bytecode

Enhanceme
nt

Store

8

Ease of Use


persistent objects definition


Persistent, Extent, @Signature,
Enhanceable,
@IgnoreSuperClass, ...


persistent fields definition


@FullText, @Identifier, @Index, @Transient, @Unique,
@Zip,
@Encrypt, ...


using Store


Application, Store, Transaction, Lookup, Condition, ...

10

ORM vs. Shards

Application

Hibernate

JDBC

SQL DB

Hibernate

Application

Shards

Shards C
-
S

Application

+

Shards

Shards emb.

11

Shards 1

class Boss implements Employee,
Persistent

{

private String name;

private String surname;

private String cv;

private int nid;


private Collection<Employee> subordinates;

}

12

Hibernate 1

@Entity(name = “Boss”)


class Boss
extends EmployeeImpl
{

private String name;

private String surname;

private String cv;

private int nid;


@OneToMany(cascade = CascadeType.ALL,
targetEntity = EmployeeImpl.class)


private Collection<Employee> subordinates;

}

13

Shards 2

class Boss implements Employee,
Persistent

{

@Identifier

long id;

private String name;

@Index
private String surname;

@FullText

private String cv;

@Unique

private int nid;


private Collection<Employee> subordinates;

}

14

Hibernate
2

@Entity(name = “Boss”) @Indexed

class Boss
extends EmployeeImpl
{

@Id @GeneratedValue @DocumentId
private Long
id;

private String name;

@Index(name = "I
1
")

private String surname;

@Field(index=Index.TOKENIZED, store=Store.NO)

private String cv;

@Column(unique=true)
private int nid;

@OneToMany(cascade = CascadeType.ALL,
targetEntity = EmployeeImpl.class)


private Collection<Employee> subordinates;

}

15

Storing and loading objects

Transaction tx = store.beginTransaction();

Employee employee = new Boss(“John”, “Smith”);

store.save(employee);

tx.commit();


tx = store.beginTransaction();

Employee employee = store.load(10);

employee.setName(“Jack”);

tx.commit();

16

Demo

17

How does it work?

Shards ClassLoader

String

Applicati
on

String

Boss

enh.

Boss

18

Enhancing


modification of bytecode before passing it to JVM


no need to write the code manually


no need to modify the code in build
-
time


optimize the code for current state of the environment


modified only classes implementing Persistent


no other classes are modified


preventing of unexpected behavior


code semantics remains unchanged


aspect oriented principle

19

Enhanced Class


lazy loading


transaction locking


reading and writing objects


indexing


reference management


...

20

Storing
100
k objects [ms]

less is better

Serializace
Shards
0
200
400
600
800
1000
1200
1400
1600
1800
2000
5 primitive
types
4 references
inheritance
21

Storing 100k objects [ms]

less is better

výchozí
burst zápis
drobné úpravy
bez indexování
0
5000
10000
15000
20000
25000
30000
35000
40000
22

Lazy Loading (LL)



loading‏of‏objects‏“when‏they‏are‏necessary”


commonly used in ODB and ORM


trying to load all necessary data


avoid unexpected behavior


trying to load no unnecessary dat


maximal performance


implementations


no LL


manual LL (DB
4
O, Darkstar)



automatic LL (Shards)


23

Žádný‏LL‏(
1
)


1

3

4

2

5

24

Max tree depth (
1
)


1

3

4

2

5

25

Objects implementing LL (
1
)


1

3

4

x

2

5

26

Objects implementing LL (
3
)


1

3

4

x

2

5

27

Objects implementing LL (
5
)


1

3

4

x

2

5

28

Full support for LL (
1
)


1

3

4

2

5

29

Full support for LL (
2
)


1

3

4

2

5

30

Full support for LL (
3
)


1

3

4

2

5

31

Full support for LL (
4
)


1

3

4

2

5

32

Full support for LL (
5
)


1

3

4

2

5

33

Predictive LL (
1
)


1

3

4

2

5

34

Persistent classes


necessary to detect access to class data


transactions


lazy loading


many ways to implement


memory consumption


performance


transparency

35

No modification


fast method calls


no modification of class


no additional memory consumption


necessary to exclusively lock on load


no lazy loading

Object

36

extends Object

Inheritance


fast method calls


no modification of original class


different class (with all original functionality)



original class can't be final


final methods make usage of inheritance useless

Object

37

Proxy
1


fast method calls


no modification of original class


original class can be final


proxy is totally different class


can't use in all cases


memory demanding

Object

Proxy

38

Proxy 2


no modification of original class


original class can be final


no bytecode manipulation


slow method calls


proxy is totally different class


can't use in all cases


memory demanding

Object

Proxy

Handler

39

Enhanced class


fast method calls


many optimization possible


it's still the original class


class signature is different

Object

40

Shadow class


fast method calls


no class modification


unified processing in Shards


necessary to exclusively lock on load


no lazy loading

Object

Shado
w

Aplikace

Shards

41

Querying in ODB


Query by Example (QbE)



querying‏for‏objects‏“similar”‏to‏specified‏object


easy to implement, easy to use


Ranged Query by Example (RQbE)



querying for objects between
2
provided objects


easy to implement, easy to use


Query API


special API for building queries (builder pattern)



easy to implement, hard to use


Native Queries (NQ)



using Java as query language


hard to implement, easy to use

42

Shards QbE

Transaction tx = store.beginTransaction();


Lookup<Employee> lookup = store.lookup();

lookup.example(new Boss(“John”, null));

Collection<Employee> es = lookup.get();


for (Employee e : es)


System.out.println(e);


lookup.delete();

tx.commit();

43

Shards NQ

Lookup<Employee> lookup = store.lookup();

lookup.match(new Condition<Employee> {

public boolean match(Employee e) {

if (!e.getName().equals(“Franta”))


return false;

if (e.getYear() <
1980
)


return true;

return isOnToBeFiredList(e);

}

});

lookup.delete();

44

Demo

45

Shards Lookup

Data

Best Employees

New
Employees

To Be Fired

Executives

Application

46

Shards Lookup


alternative to a special query language


can be pre
-
prepared


reusable


operations affect only data matching the conditions


operations on huge data sets


get, list, modify, delete, size, isEmpty


conditions


QbE, RQbE, NQ


additional functionality


sorting, paging

47

Shards Lookup

lookup.match(new Condition<Employee> {

public boolean match(Employee e) {


if (!e.getName().equals(“Franta”))


return false;


if (e.getYear() <
1980
)


return true;


return isOnToBeFiredList(e);

}

});

48

Lookup

010101
001010
101010
101010
010110
100101
010110
100101
010101
010101
010101
010010
101010
101001
010101

aload_
1

invokeinterface

iload_
0

ifne

iload_
0

ireturn

aload_
1

iload

iflt

load_
1


name == “John”

false

year <
1980

???

true

49

Lookup

John,
1970

John
,
1990

John,
1985

Bob,
1980

John,
1970

Bob,
1980

John
, 1990

John, 1985

John
,
1990

John,
1985

Shards

Condition

Result Set

51

Current market


ORM dominates


based on MS SQL, MySQL, Oracle, PostgreSQL, ...


RDBMS are highly optimized


a lot of functionality


ORM has limits of relational databases


no interfaces, problematic work with object networks, ...


several ODB attempts failed


no support by enterprises


ODB are used on small projects


limited data size, embedded devices, ...


is there some potential in ODB?

52

simplicity

scalability

DB
4
O

Hibernate

JDBC

Toplink

Shards

Caché

Serialization

53

Shards vs. DB
4
O


definition using
annotations


only serializable or
persistent objects


auto lazy
-
loading


auto update detection


garbage collection


QbE, RQbE, NQ (Lookup)



based on enhancing


under development


definition by method calls



any objects



manual lazy
-
loading


manual update notification


manual deletion


QbE, NQ, Query API


can use enhancing


stable version available

54

Mnoho prostoru na hraní


tvorba‏rozhodovaních‏stromů‏z‏byte
-
code


efektivnější‏modifikace‏tříd


metody‏zamykání‏objektů‏v‏transakcích


garbage collection


memory management


analýza‏aplikace‏využívající‏Shards


distribuovaná verze Shards


Java Validation API + Contracts (JSR
303
,
305
,
308
)



monitorovací/admin nástroje


IDE pluginy

55

Shards public alpha

Q
4 2008

Q&A


Vaclav Slovacek


vvessan@gmail.com

my.opera.com/wessan

Thank you for attention