ppt - C-JDBC

righteousgaggleΔιαχείριση Δεδομένων

31 Ιαν 2013 (πριν από 4 χρόνια και 6 μήνες)

211 εμφανίσεις

Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

1




: Flexible



Database Clustering Middleware

Emmanuel Cecchet


INRIA


Julie Marguerite


ObjectWeb

Willy Zwaenepoel


EPFL


Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

2

Internet


Database tier should be


scalable


highly available


without modifying the client application


database vendor independent


on commodity hardware

Motivations

JDBC

JDBC

JDBC

Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

3



Scaling the database tier


Alternative 1 (master
-
slave)

Internet

Web

frontend

App.

server


Cons


failover time on master failure


scalability

Master

Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

4



Scaling the database tier


Alternative 2 (SMP)

Internet

Web

frontend

App.

server

Well
-
known

database

vendor here

Database

Well
-
known hardware +

database vendors here


Cons


Cost


Scalability limit

Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

5



Scaling the database tier


Alternative 3 (shared disks)

Internet

Web

frontend

App.

server

Database

Disks

Another well
-
known

database vendor


Cons


still expensive hardware


availability

Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

6

Outline


C
-
JDBC architecture


High availability


Use cases


Conclusion

Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

7



architectural overview

JVM

Application

server

MySQL
JDBC driver

MySQL

database

JVM

C
-
JDBC
JDBC driver

C
-
JDBC

controller

JVM

Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

8



basic concepts

JVM


2 components


C
-
JDBC driver


C
-
JDBC controller


100% Java implementation


Read
-
one, Write all approach


Tunable replication


full partitioning


full replication


partial replication

Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

9

C-JDBC Controller
MySQL
C-JDBC driver
MySQL
Virtual database 1
Database
Backend
Connection
Manager
Database
Backend
Connection
Manager
Request Manager
Query result cache
Scheduler
Load balancer
MySQL
JDBC driver
MySQL
JDBC driver
Recovery
Log
Authentication Manager
MySQL
Database
Backend
Connection
Manager
MySQL
JDBC driver
C-JDBC driver
Client application
(Servlet, EJB, ...)
Client application
(Servlet, EJB, ...)
Functional overview

connect

myDB

connect

login, password

execute

SELECT * FROM t

Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

10

C-JDBC Controller
MySQL
C-JDBC driver
MySQL
Virtual database 1
Database
Backend
Connection
Manager
Database
Backend
Connection
Manager
Request Manager
Query result cache
Scheduler
Load balancer
MySQL
JDBC driver
MySQL
JDBC driver
Recovery
Log
Authentication Manager
MySQL
Database
Backend
Connection
Manager
MySQL
JDBC driver
C-JDBC driver
Client application
(Servlet, EJB, ...)
Client application
(Servlet, EJB, ...)
Functional overview

execute

INSERT INTO t …

Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

11

Outline


C
-
JDBC architecture


High availability


Use cases


Conclusion

Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

12

C-JDBC Controller
MySQL
C-JDBC driver
MySQL
Virtual database 1
Database
Backend
Connection
Manager
Database
Backend
Connection
Manager
Request Manager
Query result cache
Scheduler
Load balancer
MySQL
JDBC driver
MySQL
JDBC driver
Recovery
Log
Authentication Manager
MySQL
Database
Backend
Connection
Manager
MySQL
JDBC driver
C-JDBC driver
Client application
(Servlet, EJB, ...)
Client application
(Servlet, EJB, ...)
Failures


No 2 phase
-
commit


parallel
transactions


failed nodes are
automatically
disabled


execute

INSERT INTO t …

Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

13

Restoring a backend


Updates stored in
the recovery log


Database dumps
associated to
checkpoints

...
dump
for last
checkpoint
PostgreSQL
C-JDBC Controller
PostgreSQL JDBC driver
C-JDBC driver
JVM
EJB Container
JOnAS, WebLogic,
JBoss, WebSphere, ...
dump
for last
checkpoint
Recovery
Log
Octopus
PostgreSQL
PostgreSQL
enabled
enabled
JDBC
Recovery Log
enabled
restoring
dump
for initial
checkpoint
Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

14

Synchronization


Replay missing updates
from log

dump
for
checkpoint
Octopus
...
dump
for last
checkpoint
dump
for last
checkpoint
dump
for initial
checkpoint
PostgreSQL
C-JDBC Controller
PostgreSQL JDBC driver
C-JDBC driver
JVM
EJB Container
JOnAS, WebLogic,
JBoss, WebSphere, ...
Recovery
Log
PostgreSQL
PostgreSQL
enabled
enabled
JDBC
Recovery Log
synchronizing
enabled
Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

15

Healed Cluster


Re
-
enable backend
when done

PostgreSQL
C-JDBC Controller
PostgreSQL JDBC driver
C-JDBC driver
JVM
EJB Container
JOnAS, WebLogic,
JBoss, WebSphere, ...
dump
for
checkpoint
Recovery
Log
Octopus
PostgreSQL
PostgreSQL
enabled
enabled
JDBC
Recovery Log
enabled
enabled
...
dump
for last
checkpoint
dump
for last
checkpoint
dump
for initial
checkpoint
Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

16

Vertical scalability


Addresses JVM
scalability issues


Distributing large
number of
connections on
many backends

DB 4
DB 3
DB native JDBC driver
DB 5
C-JDBC driver
DB 1
DB 2
DB native JDBC driver
DB 6
DB native JDBC driver
DB7
C-JDBC controller
Partial replication
C-JDBC controller
Partial replication
C-JDBC controller
Partial replication
C-JDBC controller
Partial replication
C-JDBC
driver
JVM
Client
program
C-JDBC
driver
JVM
Client
program
C-JDBC
driver
JVM
Client
program
Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

17

Controller replication


Prevent the
controller from
being a single point
of failure


Group
communication for
controller
synchronization

MySQL
PostgreSQL
C-JDBC Controller
MySQL
JDBC driver
PostgreSQL
JDBC driver
C-JDBC
driver
JVM
Java client
program
C-JDBC
driver
JVM
Java client
program
C-JDBC
driver
JVM
Java client
program
C-JDBC
driver
JVM
Java client
program
C-JDBC Controller

JGroups
MySQL
PostgreSQL
MySQL
JDBC driver
PostgreSQL
JDBC driver
Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

18

Controller replication

C-JDBC Controller
Distributed Request Manager
C-JDBC Controller
Distributed Request Manager
MySQL
C-JDBC driver
MySQL
Virtual database 1
Database
Backend
Connection
Manager
Database
Backend
Connection
Manager
MySQL
JDBC driver
MySQL
JDBC driver
Virtual database 2
Database
Backend
Connection
Manager
Database
Backend
Connection
Manager
Request Manager
Query result cache
Scheduler
Load balancer
Authentication Manager
Oracle
JDBC driver
Oracle
MySQL
MySQL
JDBC driver
C-JDBC driver
Client application
(Servlet, EJB, ...)
C-JDBC driver
Client application
(Servlet, EJB, ...)
Client application
(Servlet, EJB, ...)
Request Manager
Query result cache
Scheduler
Load balancer
Authentication Manager
jdbc:cjdbc://node1,node2/myDB

Total order reliable

multicast

Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

19

Mixing horizontal & vertical
scalability

DB 6
DB 5
DB native JDBC driver
DB 7
C-JDBC driver
DB 1
DB 2
DB native
JDBC driver
DB 3
DB native
JDBC driver
DB 4
C-JDBC controller
Full replication
C-JDBC controller
Full replication
C-JDBC controller
Full replication
C-JDBC controller
Full replication
C-JDBC
driver
JVM
Client
program
C-JDBC
driver
JVM
Client
program
C-JDBC
driver
JVM
Client
program
C-JDBC
driver
C-JDBC
driver
Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

20

Current limitations


Replication granularity is table


No distributed joins


Network partition with replicated
controllers


JDBC only


support of PHP, Perl, ODBC through wrappers or bridges


partial support of JDBC 3.0


Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

21

Other features


SSL support


Support for heterogeneous databases


SQL monitoring


JMX based administration console


Request player

Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

22

Outline


C
-
JDBC architecture


High availability


Use cases


Conclusion

Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

23

Budget High Availability


High availability
infrastructure

“on a budget”


Typical eCommerce

setup




http://www.budget
-
ha.com

Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

24

OpenUSS: University Support
System


eLearning


High availability


Portability


Linux, HP
-
UX,

Windows


InterBase, Firebird,

PostgreSQL,

HypersonicSQL




http://openuss.sourceforge.net

Firebird
C-JDBC Controller
Full replication
Firebird JDBC driver
Firebird
C-JDBC driver
JVM
C-JDBC driver
JVM
C-JDBC Controller
Full replication
Firebird JDBC driver
C-JDBC driver
JVM
JOnAS
J2EE server
C-JDBC driver
JVM
Enhydra
server
OpenUSS
OpenUSS
OpenUSS
OpenUSS
Enhydra
Director
Apache
JOnAS
J2EE server
Enhydra
server
JOnAS
J2EE server
Enhydra
server
JOnAS
J2EE server
Enhydra
server
Enhydra
Director
Apache
Enhydra
Director
Apache
Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

25

Flood alert system


Disaster recovery


Independent nodes synchronized with C
-
JDBC


VPN for security issues



http://floodalert.org

MySQL
C-JDBC Controller
Full replication
MySQL JDBC driver
C-JDBC driver
JBoss
J2EE server
Flood alert
system
MySQL
C-JDBC Controller
Full replication
MySQL JDBC driver
C-JDBC driver
JBoss
J2EE server
Flood alert
system
MySQL
C-JDBC Controller
Full replication
MySQL JDBC driver
C-JDBC driver
JBoss
J2EE server
Flood alert
system
> 1000 miles
Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

26

J2EE benchmarking







Large scale

J2EE clusters



http://jmob.objectweb.org

Interne
t

emulated


users

Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

27

Outline


C
-
JDBC architecture


High availability


Use cases


Conclusion

Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

28

Conclusion


C
-
JDBC: Flexible Database Clustering
Middleware


scalable


highly available


without modifying the client application


database vendor neutral


on commodity hardware


LPGL software hosted by ObjectWeb

Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

29

Q&A

_________


Thanks to all users
and contributors ...


http://c
-
jdbc.objectweb.org

Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

30

TPC
-
W benchmark

(Amazon.com)


Nearly linear speedups with the shopping mix

0
200
400
600
800
1000
1200
1400
1600
0
1
2
3
4
5
6
Number of nodes
Throughput in requests per minute
Single Database
Full replication
Partial replication
Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

31

Result cache


Cache contains a list of SQL
-
>ResultSet


Policy defined by queryPattern
-
>Policy


3 policies


EagerCaching: variable granularities for invalidations


RelaxedCaching: invalidations based on timeout


NoCaching: never cached



RUBiS bidding mix

with 450 clients

No cache

Coherent

cache

Relaxed

cache

Throughput (rq/min)

3892

4184

4215

Avg response time

801 ms

284 ms

134 ms

Database CPU load

100%

85%

20%

C
-
JDBC CPU load

-

15%

7%

Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

32

Recovery log


All updates are
stored in the
recovery log


Database dumps
associated to
checkpoints

PostgreSQL
C-JDBC Controller
PostgreSQL JDBC driver
C-JDBC driver
JVM
EJB Container
JOnAS, WebLogic,
JBoss, WebSphere, ...
dump
for initial
checkpoint
Recovery
Log
Octopus
PostgreSQL
PostgreSQL
enabled
enabled
JDBC
Recovery Log
enabled
enabled
Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

33

Making new checkpoints


Disable one backend to
have a coherent snapshot


Mark the new checkpoint
entry in the log


Use Octopus to store

the dump

...
dump
for last
checkpoint
PostgreSQL
C-JDBC Controller
PostgreSQL JDBC driver
C-JDBC driver
JVM
EJB Container
JOnAS, WebLogic,
JBoss, WebSphere, ...
dump
for last
checkpoint
Recovery
Log
Octopus
PostgreSQL
PostgreSQL
enabled
enabled
JDBC
Recovery Log
enabled
checkpointing
dump
for initial
checkpoint
Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

34

Making new checkpoints


Replay missing
updates from log

dump
for
checkpoint
Octopus
...
dump
for last
checkpoint
dump
for last
checkpoint
dump
for initial
checkpoint
PostgreSQL
C-JDBC Controller
PostgreSQL JDBC driver
C-JDBC driver
JVM
EJB Container
JOnAS, WebLogic,
JBoss, WebSphere, ...
Recovery
Log
PostgreSQL
PostgreSQL
enabled
enabled
JDBC
Recovery Log
synchronizing
enabled
Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

35

Making new checkpoints


Re
-
enable backend
when done

PostgreSQL
C-JDBC Controller
PostgreSQL JDBC driver
C-JDBC driver
JVM
EJB Container
JOnAS, WebLogic,
JBoss, WebSphere, ...
dump
for
checkpoint
Recovery
Log
Octopus
PostgreSQL
PostgreSQL
enabled
enabled
JDBC
Recovery Log
enabled
enabled
...
dump
for last
checkpoint
dump
for last
checkpoint
dump
for initial
checkpoint
Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

36

Handling a backend failure


A node fails!


Automatically
disabled but should
be fixed or changed
by administrator

PostgreSQL
C-JDBC Controller
PostgreSQL JDBC driver
C-JDBC driver
JVM
EJB Container
JOnAS, WebLogic,
JBoss, WebSphere, ...
dump
for
checkpoint
Recovery
Log
Octopus
PostgreSQL
PostgreSQL
enabled
enabled
JDBC
Recovery Log
enabled
disabled
...
dump
for last
checkpoint
dump
for last
checkpoint
dump
for initial
checkpoint
Usenix Annual Conference, Freenix track


June 2004


c
-
jdbc@objectweb.org

37

Fault tolerant recovery log

C-JDBC Controller
MySQL
MySQL
Application virtual database
Database
Backend
Database
Backend
Request Manager
Request Cache
Scheduler
Load balancer
Recovery
Log
Authentication Manager
MySQL
Database
Backend
Recovery virtual database
Request Manager
Scheduler
Load balancer
Authentication Manager
C-JDBC driver
Client application
(Servlet, EJB, ...)
Checkpointing
service
Database
Backend
Database
Backend
C-JDBC
driver
MySQL
UPDATE statement