Alternatives of stateful replication in application servers

buninnateΛογισμικό & κατασκευή λογ/κού

18 Νοε 2013 (πριν από 3 χρόνια και 7 μήνες)

104 εμφανίσεις

Sib:

Alternatives of stateful replication
in application servers

Huaigu WU

Mcgill University

February, 2003

Purpose of Sib


An alternative approach to implement
replication in application server


Abstract level (derived from the J2EE
specification)


Focus on Stateful Session Bean


Will extend to Stateless Session Bean and
Entity Bean

Outline


Sib VS. JBoss
implementation


Simple Calls


Calling Hierarchies


Direct Database
Access



Compare


Normal process


Site crash and
failover

Application Server
Client
SFB 1
1
2
Application Server
Client
SFB 1
1
4
Data
2
3
Application Server
Client
SFB 1
1
4
SFB 2
2
3
Assumption


We don’t consider partition.


Abstract component architecture


Component in this talk consists of client, Stateful
Session Bean (SFB) and third
-
party database


Third
-
party database will be used without
modification


Deterministic component for basic analysis


One primary, one (or more) backup


So far, we assume every component is single
threaded

Normal Processing in Simple Calls
(JBoss)


Only primary receives and
processes request



Replicate primary’s up
-
to
-
date state to backups at the
end of every request before
returning its reply



Backup updates state after
every request

Backup
Primary
Client
Cluster
SFB 1
SFB 1
1
4
3
6
2
5
Normal Processing in Simple Calls (Sib)


At the beginning, states of all
replicas are the same


request is broadcast to both
primary and backup


Primary processes requests


Backup receives requests


Occasionally (periodically),
Primary’s state is replicated to
backup


Replication Point
(operational quiescence )


Independent for
component


Deterministic component:
it could be any time point

Primary
Client
Cluster
SFB 1
SFB 1
Backup
4'
1
2
3
4
0
Comparison


Network overhead for one component


Message types:


Request (req), Reply (rep), Replicated state information (s)


Message size: s >> req


JBoss analysis: s = 2
-
3 KByte


Message number (N is number of sites)


JBoss: 1 request: 1 req + 1 rep + N*s


Sib: 1 requset: N req + 1 rep + q * N * s (q is the frequency of
replication, q<=1)


Compare: JBoss > Sib


Backup’s CPU overhead


JBoss: receive and install state


Sib: receive requests


Assume install state > install request, so JBoss > Sib

Site Crash and Failover for Simple Calls

(JBoss)


Characters


When primary fails, client resends
request to the new primary


Issues


Resend must be repeatable


Content


Duplicated requests must be
identified by SFB

Cluster
Client
SFB 1
Primary
1
SFB 1
Backup
Resend 1
3
2
Cluster
Client
SFB 1
Primary
1
SFB 1
Backup
Resend 1
3
2
Cluster
Client
SFB 1
Primary
1
SFB 1
Backup
2
Resend 1
3
Identify duplicated request 1
(Does JBoss do it ?)
Site Crash and Failover for Simple Calls

(Sib)


Characters


When primary fails, the new
primary automatically re
-
executes all requests starting
from last replication point


Client does not need to resend
request



Issues


Duplicated replies must be
identified by client


Replies must be deterministic

Cluster
Client
SFB 1
Primary
SFB 1
Backup
2
2
Identify duplicated reply 2
(testable result)
Execute
request 1
1
Cluster
Client
SFB 1
Primary
SFB 1
Backup
2
Execute
request 1
1
Normal Processing in Calling Hierarchies

JBoss

Sib

Backup
Primary
Client
Cluster
SFB 1
SFB 1
SFB 2
SFB 2
1
6
2
4
3
5
Backup
Primary
Client
Cluster
SFB 1
SFB 1
SFB 2
SFB 2
4
3
1
2
0
0
3'
Site Crash and Failover for Calling
Hierarchies (JBoss)


Assumption


Node fails (all beans in the node fail)


Characters


Similar to simple calls for one
component


Issues


Duplicated internal
requests must be
identified by SFB

Backup
Primary
Client
Cluster
SFB 1
SFB 1
SFB 2
SFB 2
1
2
Resend 1
6
3
4
5
Backup
Primary
Client
Cluster
SFB 1
SFB 1
SFB 2
SFB 2
1
2
Resend 1
2
6
5
3
4
Duplicated
internal
request 2
4
Backup
Primary
Client
Cluster
SFB 1
SFB 1
SFB 2
SFB 2
1
2
Resend 1
2
4
3
6
5
Site Crash and Failover for Calling
Hierarchies (Sib)


Assumption:


Node fails (all beans in the node fail)


Character:


Every backup SFB re
-
executes
requests starting from its own
last replication point

Backup
Primary
Client
Cluster
SFB 1
SFB 1
SFB 2
SFB 2
4
3
7
7
6
3
2
8
3'
Duplicated
reply 8
2
6
0
0
Discard
duplicated
internal
requests
Identify
duplicated
internal
requests
Duplicated
reply 4
1
5

Issues


Duplicated internal requests
must be identified by SFB


Duplicated internal replies
must be identified by SFB

Backup
Primary
Client
Cluster
SFB 1
SFB 1
SFB 2
SFB 2
4
3
7
7
8
Duplicated
reply 8
1
5
2
6
0
0
4'
Re-execute
request 2,6
Re-execute
request 5
3
6
Duplicated
internal
request
Duplicated
internal
replies
Backup
Primary
Client
Cluster
SFB 1
SFB 1
SFB 2
SFB 2
4
3
7
7
8
Duplicated
reply 8
1
5
2
6
0
0
4'
Re-execute
request 6
Re-execute
request 5
6
Duplicated
internal
request
3'
Direct Database Access (JBoss)


Normal process

Cluster
Client
SFB 1
Primary
1
SFB 1
Backup
Data
Write and commit
Ack
Resend 1
3
2
Possible duplicated
database access
Write and commit

Failover


NO “Exactly
-
once” Semantic

Cluster
Client
SFB 1
Primary
1
SFB 1
Backup
Data
Write and commit
2
3
Ack
Resource Manager
(Resource Adapter)
Direct Database Access (Sib)


Normal process


Data bind to specified
Resource Manager



All requests to some
data is forwarded to its
specified RM



Focus on write
operation

Data
Backup
Primary
Client
Cluster
SFB 1
SFB 1
RM
RM
2
Ack
1
0
0
Write & Commit
SQL
Ack
Direct Database Access (Sib)


Failover suggestion 1


Characters


Write bind to commit


Converse Undo, start from
the last write.


If original write is successful,
undo rollback it.


If original write is failed,
undo has no effect.


Issues


Tables must have primary
key to be able to generate
undo


In case primary committed,
other access between
commit and undo might be
possible


Conclusion


Simple during normal
process


Complex during recovery

Data
Backup
Primary
Client
Cluster
SFB 1
SFB1
RM
RM
1
0
0
Write &
Commit
Undo to
rollback
(repeatable)
Ack
Write & Commit
Ack
Ack
SQL
2
Re-execute
request 1
Duplicated internal
SQL request
Re-execute
SQL request
SQL
Generate
undo
Direct Database Access (Sib)


Failover Suggestion 2


Characters


Insert Additional mark
before every commit


If mark exists, redo is not
required


If mark does not exist,
redo


Issues


additional insert
overhead In normal
process


RM must maintain global
transaction ids


Conclusion


Complex during normal
process


Simple and correct
during recovery

Data
Backup
Primary
Client
Cluster
SFB 1
SFB1
RM
RM
1
0
0
Write
Ack
Redo,
if no mark
Ack
SQL
2
Re-execute
request 1
Duplicated internal
SQL request
SQL
Insert
Mark
commit
Check Mark
Mark
Exist?
Summary


Sib’s Advantage


Better performance


State information (including response information) is much
bigger than request


Infrequent replication points enough since EJB is short
-
living
so that failure is unlikely


More powerful for “Exactly
-
once” database access


Sib’s Disadvantage


More complicated recovery


Duplicated external responses


Client needs to remove duplicated responses

Current Work


Compare schemas for complex transactions


A transaction crosses several requests


Entity beans replication in complex transactions


Multi
-
Thread: Typical Non
-
deterministic environment


Multi
-
Thread Client / Single
-
Thread SFB


JBoss:


Repeatable request might be problematic (e.g. resend in different order)


Sib


Avoid resend requests


Total order o requests in all replicas


Multi
-
Thread Client / Multi
-
Thread SFB


Both JBoss and Sib might be problematic


Sib: special concurrency control and synchronous mechanism


Component Failure

Problematic Example under Multi
-
Thread Environment


Cluster
Client
Primary
1
Backup
Resend 2
2
2
X+3
X-3
Resend 1
S2
X=0
X=3
X=-3
X=0
S2

S
1

X
-
3

Reference


[1] W.Zhao, L.E.Moser and P.M.Melliar
-
smith, “Unification of Replication and Transaction
Processing in the Three
-
Tier Architectures”,
22nd International Conference on Distributed
Computing Systems (ICDCS'02)
, Vienna, Austria, July 02
-

05, 2002.


[2] Marcia Pasin, Michel Riveill and Taisy Silva Weber, “High
-
Available Enterprise
JavaBeans Using Group Communication System Support”, Proceedings of the Fourth
European Research Seminar on Advances in Distributed Systems
-

ERSADS 2001, pages
161
-
166, University Residential Center of University of Bologna, Berliner (Frolic), Italy, 14
-
18 May 2001.


[3] Roger Barga, David Lomet, Gerhard Weikum, “Recovery Guarantees for General Multi
-
Tier Aplications”, Proceedings of the 18th International Conference on Data Engineering
(ICDE'02)


[4] P.Narasimhan, L.E.Moser and P.M.Melliar
-
smith, “State Synchronization and Recovery
for Strongly Consistent Replicated CORBA Objects”, International Conference on
Dependable Systems and Networks, Göteborg, Sweden (July 2001).


[5] N.Narasimhan, L.E.Moser and P.M.Melliar
-
smith, “Transparent Consistent Replication
of Java RMI Object”,
DOA 2000
: 17
-
26


[6]
Svend Frølund
, Rachid Guerraoui, “e
-
Transactions: End
-
to
-
End Reliability for Three
-
Tier Architectures”,
TSE 28
(4): 378
-
395 (2002)


[7]
Svend Frølund
, Rachid Guerraoui, “Implementing e
-
Transactions with Asynchronous
Replication”,
IEEE Transactions on Parallel and Distributed Systems 12
(2): 133
-
146 (2001)


[8] Object Management Group, “Fault Tolerant CORBA Specification”. December 1999,
OMG Technical Committee Document (orbos/00
-
04
-
04)

Reference


[9] IONA Technologies PLC, “White Paper Orbix E2A Application Load Balancing and
Fault Tolerance”, online documentation,
http://www.iona.com/forms/wprequest.htm
,
April, 2002.


[10] BEA WebLogic Server™, “Programming WebLogic Enterprise JavaBeans”,
Copyright

© 2000 BEA Systems, Inc. All rights reserved.


[11] Gerhard Weikum, Gpttfried Vossen, “Transactional Information Systems: Theory.
Algorithms, and the Practice of Concurrency Control and Recover”, Morgan
Kaufmann Publishers, January 2002.


[12] Ed Roman, Scott Ambler and Tyler Jewell, “Mastering Enterprise JavaBeans
second edition”, Wiley Computer Publishing, John Wiley & Sons, Inc., ISBN: 0
-
471
-
41711
-
4, December 2001.


[13] Sun Microsystems Inc., “Java™ 2 Platform Enterprise Edition Specification, v1.3”,
online documentation,
http://java.sun.com/j2ee/
, Oct. 20, 2000.


[14] Sun Microsystems Inc., “Enterprise JavaBeansTM Specification, Version 2.0”,
online documentation,
http://java.sun.com/j2ee/
, Oct. 23, 2000.


[15] R. Jimenez
-
Peris, M. Patino
-
Martinez, “Deterministic Scheduling and Online
Recovery for Replicated Multithreaded Transactional Servers”, WDMS 2002, June 26,
2002.


[16] Roman Vitenberg, Idit Keidar, Gregory V.Chockler and Danny Dolev, “Group
Communication Specifications: A Comprehensive Study”, In ACM Computing
Surveys 33(4), pages 1
-
43, December 2001.


[17] Sacha Labourey, Bill Burke, “JBoss Clustering”, The JBoss Group, November
2002


[18] Andreas Schaefer, “JBoss 3.0 Quick Start Guide”, The JBoss Group, July 2002