Clustering the JVM using AOP

Jonas Bonér

Terracotta, Inc.

jonas@terracotta.org

http://terracotta.org

http://jonasboner.com


Clustering the JVM using AOP



Let’s start with a demo


–

since a picture says more than a thousand words

Agenda


Overview of Terracotta


Get started


The inner workings of a clustered JVM


Real
-
world examples


Q&A

Agenda


Overview of Terracotta


Get started


The inner workings of a clustered JVM


Real
-
world examples


Q&A

What is Terracotta?

Network
-
Attached Memory

Open Source Scalability and

High
-
Availability for the JVM

What is Terracotta?


Network
-
Attached Memory


No API
: declarative configuration to
selectively share object graphs across
the cluster


No serialization
: plain POJO clustering


Fine
-
grained replication
:
field
-
level, heap
-
level replication


Cross
-
JVM coordination
: Java Memory Model maintained across the
cluster
–

e.g. cluster
-
wide
wait/notify

and
synchronized


Large virtual heaps
:

that exceeds the capacity of a single JVM


Distributed Method Invocations



Runtime
monitoring and control

Use
-
cases


Relieving Database Overload

•
Distributed Caching

•
Hibernate Clustering

•
HTTP Session Clustering


Simplifying Application Architecture and Development

•
Virtual Heap for Large Datasets

•
Clustering OSS Frameworks (Spring, Struts, Lucene, Wicket,
EHCache etc.)

•
Master/Worker
–

Managing Large Workloads

•
POJO Clustering

•
Messaging, Event
-
based Systems and Coordination
-
related Tasks

Terracotta approach


Today's Reality

•
Scale out is complex

•
Requires custom Java code


Our different approach

•
Cluster the JVM

•
Eliminate need for custom code

Scale
-
out



Terracotta Server

Clustering the JVM

App Server

Web App

JVM

Frameworks

Frameworks

Business Logic

App Server

Web App

JVM

Frameworks

Frameworks

Business Logic

App Server

Web App

JVM

Frameworks

Frameworks

Business Logic

Custom Development

JMS

RMI

Serialization

The architecture of Terracotta


Terracotta Server

•
100% Pure Java

•
HA Active / Passive Pair


Local JVM Client

•
Transparent

•
Pure Java Libraries


Coordinator “traffic cop”

•
Coordinates resource access

•
Runtime optimizations


Central Storage

•
Maintains state across restarts


Terracotta Server (PASSIVE)

Clustering the JVM

Scale
-
out

Terracotta Server (ACTIVE)

Clustering the JVM

App Server

Web App

JVM

Any Java Objects

Any Framework

Sessions Objects

App Server

Web App

JVM

Any Java Objects

Any Framework

Sessions Objects

App Server

Web App

JVM

Any Java Objects

Any Framework

Sessions Objects

Terracotta Library

Terracotta Library

Terracotta Library

Terracotta Server

Clustering the JVM


Heap Level Replication

•
Declarative

•
No Serialization

•
Fine Grained / Field Level


GET_FIELD

-

PUT_FIELD

•
Only Where Resident


JVM Coordination

•
Distributed Synchronized
Block

•
Distributed
wait()/notify()

•
Fine Grained Locking
MONITOR_ENTRY
-
MONITOR_EXIT



Large Virtual Heaps

•
As large as available disk

•
Dynamic paging

Scale
-
out

App Server

Web App

JVM

TC Libraries

Shared

Objects

Terracotta Server

Clustering the JVM

App Server

Web App

JVM

TC Libraries

Shared

Objects

App Server

Web App

JVM

TC Libraries

Shared

Objects

Network
-
Attached

Memory

TC

Management

Console


Management Console

•
Runtime visibility

•
Data introspection

•
Cluster monitoring

Agenda


Overview of Terracotta


Get started


The inner workings of a clustered JVM


Real
-
world examples


Q&A

Hello World

Hello World
-

tutorial/HelloWorld.java

package

tutorial
;


import

java
.
util
.
*
;


public

class

HelloWorld

{


private

List
<
String
>

hellos

=

new

ArrayList
<
String
>
();



public

void

sayHello
()

{


synchronized
(
hellos
)

{


hellos
.
add
(
"Hello,

World

"

+

new

Date
());


for
(
String

hello

:

hellos
)

{


System
.
out
.
println
(
hello
);


}


}


}



public

static

void

main
(
String
[]

args
)

{


new

HelloWorld
().
sayHello
();


}

}

Hello World

-

tc
-
config.xml

<?xml

version="1.0"

encoding="UTF
-
8"?>

<
tc:tc
-
config

xmlns:tc
=
"http://www.terracotta.org/config"
>


<
application
>


<
dso
>


<
roots
>


<
root
><
field
-
name
>
tutorial.HelloWorld.
hellos
</
field
-
name
></
root
>


</
roots
>


<
locks
>


<
autolock
>


<
method
-
expression
>
*

tutorial.HelloWorld*.*(..)
</
method
-
expression
>


<
lock
-
level
>
write
</
lock
-
level
>


</
autolock
>


</
locks
>


<
instrumented
-
classes
>


<
include
><
class
-
expression
>
tutorial..*
</
class
-
expression
></
include
>


</
instrumented
-
classes
>


</
dso
>


</
application
>

</
tc:tc
-
config
>

Demo

Enhanced Hello World

Using java.util.concurrent

Coordination

-

HelloWorldConcurrent.java

package

tutorial
;


import

java
.
util
.
*
;

import

java
.
util
.
concurrent
.
CyclicBarrier
;


public

class

HelloWorldConcurrent

{


private

List
<
String
>

hellos

=

new

ArrayList
<
String
>
();


private

CyclicBarrier

barrier
;



public

void

sayHello
()

throws

Exception

{


barrier

=

new

CyclicBarrier
(
2
);


barrier
.
await
();



synchronized
(
hellos
)

{


hellos
.
add
(
"Hello,

World

"

+

new

Date
());


for
(
String

hello

:

hellos
)

{


System
.
out
.
println
(
hello
);


}


}


}



public

static

void

main
(
String
[]

args
)

throws

Exception

{


new

HelloWorldConcurrent
().
sayHello
();


}

}


Coordination

-

tc
-
config
-
concurrent.xml

<?xml

version="1.0"

encoding="UTF
-
8"?>

<
tc:tc
-
config

xmlns:tc
=
"http://www.terracotta.org/config"
>


<
application
>


<
dso
>


<
roots
>


<
root
><
field
-
name
>
tutorial.HelloWorldConcurrent.hellos
</
field
-
name
></
root
>


<
root
><
field
-
name
>
tutorial.HelloWorldConcurrent.
barrier
</
field
-
name
></
root
>


</
roots
>


<
locks
>


<
autolock
>


<
method
-
expression
>
*

tutorial.HelloWorldConcurrent*.*(..)
</
method
-
expression
>


<
lock
-
level
>
write
</
lock
-
level
>


</
autolock
>


</
locks
>


<
instrumented
-
classes
>


<
include
><
class
-
expression
>
tutorial..*
</
class
-
expression
></
include
>


</
instrumented
-
classes
>


</
dso
>


</
application
>

</
tc:tc
-
config
>

Demo

Agenda


Overview of Terracotta


Get started


The inner workings of a clustered JVM


Real
-
world examples


Q&A

Inner workings of a clustered JVM


Maintain the semantics of the application (e.g Java
Language Specification (JLS) and the Java Memory
Model (JMM)) across the cluster


Load
-
time bytecode instrumentation

•
Hook in
java.lang.ClassLoader

and
-
javaagent

•
Transparent to the user
--

classes are woven on
-
the
-
fly


Aspect
-
Oriented Programming techniques

•
AspectWerkz aspects

•
Custom bytecode instrumentation using ASM

•
Declarative configuration (XML)


Simplified pointcut pattern language (based on AspectWerkz)


What do we need to maintain across the
cluster?

1.
State sharing

•
Share data in the Java heap for the instances reachable from a
shared top
-
level “root” reference

•
Make sure each JVMs local heap are in sync (in regards to the
clustered parts of the heap)

2.
Thread coordination

•
Resource access and guarding

•
Thread signaling

Sample application


@Clustered public class Counter {



@Root public final static Counter instance = new Counter();



private final Object lock = new Object();



private volatile int counter = 0;




public void increment() {



synchronized(lock) { this.counter++; lock.notifyAll(); }



}



public void waitFor(int expected) {



synchronized(lock) {



while(this.counter < expected) {



try { lock.wait(); } catch(InterruptedException ex) {}



}



}



}

put top level root in shared space

lock and start tx

commit tx and unlock

reconcile pending changes

record field change

intercept notifyAll

instrument clusterable classes

short circuit root creation

State sharing
-

field
-
level replication and
management


We need to break down the shared object graph into
“literals”

•
int, long, float, String, boolean etc.


Need to be able to detect changes to the object graph

•
New object reference is added/removed

•
Literals changes value


Every reference has a “shadow” that is “managing” the
reference

•
Knows if the reference has been changed

•
Can lazily page in and out the actual value

•
Page in: Reconcile changes done in another node

•
Page out: Null out the reference


Field
-
level replication

Child2

Child1

Child3

Parent2

500 bytes

500 bytes each

16 bytes each


Total object graph size: 1548 bytes

Grandparent

??

Parent1

each instance has a shadow

that is managing access

and modification

Child3


Terracotta replicates: 16 bytes

Manage field access and modification


Intercept the
PUTFIELD

and
GETFIELD

bytecode instructions


Create two advice for managing get and set shared field


•
before(TransparentAccess o, Object value) :

•

set(* *.*)

&& isClusteredTarget(o) &&
args(value)

{

•

… // register changed field value in shadow

•
}


•
before(TransparentAccess o) :

•

get(* *.*)

&& isClusteredTarget(o) {

•

… // reconcile any pending changes
?

•
}


BCI Example
–

original byte code


public demo.sharededitor.controls.Dispatcher(demo.sharededitor.models.ObjectManager,
demo.sharededitor.ui.Renderer);



Code:



0:

aload_0



1:

invokespecial

#1; //Method javax/swing/event/MouseInputAdapter."<init>":()V



4:

aload_0



5:

aload_2



… snip



33:

aload_0



34:

aload_1



35:

putfield

#6; //Field objmgr:Ldemo/sharededitor/models/ObjectManager;



38:

aload_0



39:

getfield

#6; //Field objmgr:Ldemo/sharededitor/models/ObjectManager;



42:

aload_0



43:

getfield

#2; //Field renderer:Ldemo/sharededitor/ui/Renderer;



46:

invokevirtual

#7; //Method
demo/sharededitor/models/ObjectManager.addListener:(Ldemo/sharededitor/events/IListListener;)V



49:

return

BCI Example
–

instrumented byte code


public demo.sharededitor.controls.Dispatcher(demo.sharededitor.models.ObjectManager,
demo.sharededitor.ui.Renderer);



Code:



0:

aload_0



1:

invokespecial

#99; //Method javax/swing/event/MouseInputAdapter."<init>":()V



4:

aload_0



5:

aload_2



… snip



33:

aload_0



34:

aload_1



35:

invokevirtual

#117; //Method __
tc_setobjmgr
:(Ldemo/sharededitor/models/ObjectManager;)V



38:

aload_0



39:

invokevirtual

#119; //Method __
tc_getobjmgr
:()Ldemo/sharededitor/models/ObjectManager;



42:

aload_0



43:

getfield

#101; //Field renderer:Ldemo/sharededitor/ui/Renderer;



46:

invokevirtual

#123; //Method
demo/sharededitor/models/ObjectManager.addListener:(Ldemo/sharededitor/events/IListListener;)V



49:

return

Thread coordination


We need to maintain the semantics of the Java Memory
Model (JMM) across multiple JVMs


Maintain the semantics of:

•
synchronized(object) {..}

•
wait()/notify()/notifyAll()

•
Flushing of changes



Locking optimizations

•
User defined

•
Runtime optimized

Manage synchronized blocks



Intercept
MONITORENTER

and
MONITOREXIT

bytecode
instructions


before(Object o) :



isClustered() &&
lock()

&&
args(o)

{



ClusterManager.lock(o); // lock o in cluster


}


after(Object o) : // after finally



isClustered() &&
unlock()

&&
args(o)

{



ClusterManager.unlock(o); //unlock o in cluster + flush
changes


}


Manage thread coordination (wait/notify)


Intercept the
wait()
,
notify()

and
notifyAll()

methods in
java.lang.Object



void around(Object o) :



isClusteredTarget(o) &&
call(void Object.wait())

{



ClusterManager.objectWait(o);



}



void around(Object o) :



isClusteredTarget(o) &&
call(void Object.notify())

{



ClusterManager.objectNotify(o);



}



… // remaining methods omitted

Agenda


Overview of Terracotta


Get started


The inner workings of a clustered JVM


Real
-
world examples


Q&A

HTTP Session Clustering

without Java Serialization

HTTP Session Clustering

-

Benefits


Terracotta Sessions gives you:

•
Near
-
Linear Scale

•
No
java.io.Serializable

•
Large Sessions
-

MBs

•
Higher Throughput


Supported Platforms:

•
Jetty,

•
JBoss 4.x,

•
Tomcat 5.0 & 5.5,

•
WebLogic 8.1, WebLogic 9.2,

•
WebSphere CE, Geronimo Alpha, WebSphere 6.1

HTTP Session Clustering

-

DummyCart.java

package

demo
.
cart
;



import

java
.
util
.
*
;



public

class

DummyCart

{


private

List

items

=

new

ArrayList
();




public

List

getItems
()

{


return

Collections
.
unmodifiableList
(
items
);


}




public

void

addItem
(
String

name
)

{


items
.
add
(
name
);


}




public

void

removeItem
(
String

name
)

{


items
.
remove
(
name
);


}

}

HTTP Session Clustering

-

carts.jsp

<%@
page
import
="
java.util.Iterator
" %>

<
html
>


<
jsp:useBean
id
=
"
cart
"
scope
=
"
session
"
class
=
"
demo.cart.DummyCart
"
/>


<%
String submit = request.getParameter(
"submit"
);


String item = request.getParameter(
"item"
);


if
(submit !=
null
&& item !=
null
) {


if
(submit.equals(
"add"
)) {


cart.addItem(item);


}
else if
(submit.equals(
"remove"
)) {


cart.removeItem(item);


}


}
%>



<
body
>


<
p
>You have the following items in your cart:</
p
>


<
ol
>
<%



Iterator it = cart.getItems().iterator();


while
(it.hasNext()) {


%>
<
li
>
<%
out.print(it.next());
%>
</
li
>
<%



}
%>



</
ol
>




<
form
method=
“get"
action=
"
<%=
request.getContextPath()
%>
/cart.jsp
"
>


<
p
>Item to add or remove: <
input
type=
"text"
name=
"item"
/></
p
>


<
input
type=
"submit"
name=
"submit"
value=
"add"
/>


<
input
type=
"submit"
name=
"submit"
value=
"remove"
/>


</
form
>


</
body
>

</
html
>

HTTP Session Clustering

-

tc
-
config.xml

<?xml

version="1.0"

encoding="UTF
-
8"?>

<
tc:tc
-
config

xmlns:tc
=
"http://www.terracotta.org/config"
>


<
application
>


<
dso
>



<
web
-
applications
>



<
web
-
application
>
cart
</
web
-
application
>


</
web
-
applications
>


<
instrumented
-
classes
>


<
include
>


<
class
-
expression
>
demo.*
</
class
-
expression
>


</
include
>


</
instrumented
-
classes
>


</
dso
>


</
application
>

</
tc:tc
-
config
>

Demo

Cluster Spring and

other OSS frameworks


'Supported'

by Terracotta means:

•
Just include a ‘Configuration Module’ (OSGi bundle) in your config to
cluster your application (a one liner)

•
Plugs in underneath without any setup

•
Technically, Terracotta supports all integrations as long as it runs on the
JVM (f.e. JRuby, PHP)


Glimpse of supported integrations

Example Configuration Module

-

EHCache

clustering

<?xml

version="1.0"

encoding="UTF
-
8"?>

<
tc:tc
-
config

xmlns:tc
=
"http://www.terracotta.org/config"
>


<
clients
>


<
modules
>



<
module

name
=
"clustered
-
ehcache
-
1.3"

version
=
"1.0.0"
/>


</
modules
>


</
clients
>


<
application
>


<
dso
>


<
instrumented
-
classes
>


<
include
>


<
class
-
expression
>
tutorial.*..*
</
class
-
expression
>


</
include
>


</
instrumented
-
classes
>


</
dso
>


</
application
>

</
tc:tc
-
config
>

Example
-

Spring clustering



Terracotta can declaratively cluster Spring beans (Singleton +
Session and Custom scoped) with zero code changes


Can also cluster Spring ApplicationContext events, JMX State and
Spring Web Flow

<
spring
>


<
application

name
=
"tc
-
jmx"
>


<
application
-
contexts
>



<
application
-
context
>


<
paths
>


<
path
>
*/applicationContext.xml
</
path
>


</
paths
>


<
beans
>


<
bean

name
=
"clusteredCounter"
/>


<
bean

name
=
"clusteredHistory"
/>


</
beans
>


</
application
-
context
>


</
application
-
contexts
>



</
application
>

</
spring
>

<
bean

id
=
"localCounter"



class
=
"demo.jmx.Counter"
/>



<
bean

id
=
"clusteredCounter"



class
=
"demo.jmx.Counter"
/>



<
bean

id
=
"localHistory"



class
=
"demo.jmx.HistoryQueue"
/>



<
bean

id
=
"clusteredHistory"



class
=
"demo.jmx.HistoryQueue"
/>

Terracotta config

Spring config

Demo


Cluster Spring Web Flow’s

Web Continuations

(conversational/workflow state)


Agenda


Overview of Terracotta


Get started


Real
-
world examples


Q&A

Wrapping up

Wrapping up


Terracotta is Network
-
Attached Memory for the JVM


Turns Scalability and High
-
Availability into a
deployment artifact



Keep the simplicity of POJO
-
based development
–

get
Scale
-
Out with Simplicity


Makes mission
-
critical applications
simpler to
:

•
Write

•
Understand

•
Test

•
Maintain


Endless possibilities for clustering and distributed programming
–

these were just a few


Be creative, use your imagination and have fun…

Questions?




http://terracotta.org

jonas@terracotta.org