2009 Oracle Corporation - Oracle Coherence Knowledge Base Home

squawkpsychoticSoftware and s/w Development

Dec 2, 2013 (3 years and 9 months ago)

91 views

© 2009 Oracle Corporation
© 2009 Oracle Corporation
The Portable Object Format
Coherence 3.5
© 2009 Oracle Corporation
The following is intended to outline our general
product direction. It is intended for information
purposes only, and may not be incorporated into any
contract. It is not a commitment to deliver any
material, code, or functionality, and should not be
relied upon in making purchasing decisions.
The development, release, and timing of any
features or functionality described for Oracle’s
products remains at the sole discretion of Oracle.
© 2009 Oracle Corporation
The Portable Object Format
(advanced serialization)


Introduction


Platform and Language independent format for representing
objects


Designed in 2006 and part of Coherence since 3.2


First used to support .NET Client over *Extend


Then used to support C++ Client over *Extend


Now the
preferred
serialization approach for all languages
(including Java)


Why?


Complete interoperability (including versioning)


Performance (faster) and Size (on the wire and heap)
© 2009 Oracle Corporation
The Portable Object Format
(advanced serialization)



But what about Java Serialization and
ExternalizableLite?


Both are still supported


Recommendation


Start to consider PortableObject over ExternalizableLite


One issue:



Once you move to Portable Object, it’s currently hard to go
back
(without implementing dual interfaces)
© 2009 Oracle Corporation
The Portable Object Format
(advanced serialization)



Portable Object v’s ExternalizableLite



Implement PortableObject interface
(inste
a
d of ExternalizableLite)



Similar readExternal and writeExternal methods
(but easier to use

no ExternalizableHelper required)


public void readExternal(PofReader reader) { … }


public void writeExternal(PofWriter writer) { … }


No need to implement java.io.Serializable support
(as of Coherence 3.4 DP2)



May
avoid implementing PortableObject by using customized
Serialization ie: Clean PO?OS… no serialization interfaces!
© 2009 Oracle Corporation
The Portable Object Format
(advanced serialization)



Design Goals
(requirements)



Unambiguous binary format for Object and Method Invocation
(Portable Invocation Format
=
PIF)


Self-describing to permit
traversal
of the POF stream


Support representations of all common value types, including
data-structures and
user types



Support versioning to allow
addition
of information over time


Supports forward and backward compatibility


Extremely compact in representation
© 2009 Oracle Corporation
The Portable Object Format
(advanced serialization)



Design Goals
(requirements)



Language neutral


Support C++ on Linux, Solaris, Windows, OS X


Support C# and .NET on Windows


Support direct language to language communication
(without going via a Data Grid)


Support invoking a method against one interface, but
implemented in another
(ie: PIF)

© 2009 Oracle Corporation
The Portable Object Format
(advanced serialization)



Non-Design Goals
(non requirements)



Human readability


Compatibility with existing specifications (like ASN1, Hessian)


Support for all Java types
(primitive and collection types supported)


Support for all user types


Support for all object relational types


Completely self-describing
© 2009 Oracle Corporation
The Portable Object Format
(advanced serialization)



Memory & Wire Format


Stream of octet values
(8-bit unsigned bytes: 0x00 to 0xFF)


Byte-ordering is well-defined
(irrespective of platform)


POF Stream


Contains a single
POF Value


PIF Stream


Conversational Identifier


POF Value
(representing the Method Invocation)
© 2009 Oracle Corporation
The Portable Object Format
(advanced serialization)



POF Value


Contains
POF Type Identifier
(represented as a POF integer)


(optionally)
Followed by octets representing encoded value of
the specified
Type Identifier


Some
POF Type Identifiers
are “values”


boolean:true, boolean:false, string:empty, reference:null
collection:empty, floating-point:+infinity etc…


Length of a POF Type Identifier is
not restricted
to a single
octet
(integers are stored in variable-length packed binary format)
© 2009 Oracle Corporation
The Portable Object Format
(advanced serialization)



POF Integers


Relies on ability to encode values in a compact form
(packed binary format)



First octet contains


1 bit: “negative” indicator


6 bits: least-significant-bits of the value


1 bit: “continuation” indicator
(ie: is there more to follow)



Remaining octets contain


7 bits: next-significant-bits of the value


1 bit: “continuation” indicator
(ie: is there more to follow)
© 2009 Oracle Corporation
The Portable Object Format
(advanced serialization)



Simple Serialization Comparison


In XML


<date format=“java.util.Date”>2009-07-03</date>


47 characters
(possibly 94 bytes depending on encoding)


In Java
(as a raw long)


64 bits = 8 bytes


In Java
(java.util.Date using ObjectOutputStream)


46 bytes


In ExternalizableLite
(as a raw long)


8 bytes


In POF


4F 58 1F 70 6C = 5 bytes
© 2009 Oracle Corporation
The Portable Object Format
(advanced serialization)



Native POF Types


(Type Ids are encoded as Negative Integers)


int16, int32, int64, int128*, float32, float64, float128*,
decimal32*, decimal64*, decimal128*, boolean, octet, octet-
string, char, char-string, date, year-month-interval*, time,
time-interval*, datetime, day-time-interval*, collection,
uniform-collection, array, uniform-array, sparse-array,
uniform-sparse-array, map, uniform-keys-map, uniform-map,
identity, referenced


“uniform” collections = have common element type
© 2009 Oracle Corporation
The Portable Object Format
(advanced serialization)



User Type Encoding


POF Type Identifier
(encoded as a positive integer)


not a “class-name”


The “meaning” of the Type Identifier is specified by a
Context



Version Indicator
(encoded as a positive integer)


A sequence of zero or more
(Property Id
,
POF Value)
pairs


Terminator (the value of -1)


Coherence Internal Types Identifiers


Reserved Range to 1000
© 2009 Oracle Corporation
The Portable Object Format
(advanced serialization)



Steps to use Portable Objects (in Java)
1.

Implement
com.tangosol.io.pof.PortableObject



For classes of objects placed in a Cache


For custom EntryProcessors, Filters, Extractors
Aggregators and Invocables
2.

Use the PofWriter and PofReader to serialize & deserialize
3.

Declare POF User Types in pof-config.xml
4.

Configure cache-schemes to use POF Serializer
5.

-Dtangosol.pof.config=pof-config.xml
6.

Deprecate other Serialization implementations


© 2009 Oracle Corporation
POF Example
Position.java
(partial)

import java.io.IOException;
import com.tangosol.io.pof.PofReader;
import com.tangosol.io.pof.PofWriter;
import com.tangosol.io.pof.PortableObject;
/**
* An immutable class to represent a single position in a financial
* market for an equity (stock)
*/
public class
Position implements
PortableObject
{

private int id;
private String symbol;
private int amount;
private double price;
© 2009 Oracle Corporation
POF Example
Position.java
(partial)


/**


* Required for PortableObject

*/

public Position() {

}



/**


* The standard constructor for a {@link Position}
*/
public Position(int id, String symbol, int amount, double price) {
this.id = id;
this.symbol = symbol;



this.amount = amount;



this.price = price;


}
© 2009 Oracle Corporation
POF Example
Position.java
(partial)

public void readExternal(PofReader reader) throws IOException {
id = reader.readInt(0);
symbol = reader.readString(1);
amount = reader.readInt(2);
price = reader.readDouble(3);


}
public void writeExternal(PofWriter writer) throws IOException {
writer.writeInt(0, id);
writer.writeString(1, symbol);
writer.writeInt(2, amount);
writer.writeDouble(3, price);
}
The Property Index
© 2009 Oracle Corporation
POF Example
pof-config.xml
(complete)

<pof-config>


<user-type-list>




<!-- include all of the standard Coherence type -->
<include>coherence-pof-config.xml</include>




<!-- our custom types (user types) -->



<user-type>



<type-id>1000</type-id>


<class-name>Position</class-name>



</user-type>
</user-type-list>



</pof-config>


© 2009 Oracle Corporation
POF Example
cacheconfig.xml
(partial)

<caching-scheme>
<distributed-scheme>



<scheme-name>distributed-scheme</scheme-name>


<service-name>DistributedCache</service-name>
<backing-map-scheme>
<local-scheme>
</local-scheme>
</backing-map-scheme>



<serializer>
<class-name>com.tangosol.io.pof.ConfigurablePofContext</class-name>
</serializer>
<autostart>true</autostart>
</distributed-scheme>
</caching-scheme>


© 2009 Oracle Corporation
The Portable Object Format
(advanced serialization)



Evolvable


POF can support forward and backward serialization
compatibility



This is defined by the
EvolvablePortableObject
interface


public interface EvolvablePortableObject
extends PortableObject, Evolvable
© 2009 Oracle Corporation


The
Evolvable
Interface

public interface Evolvable
{
public int getImplVersion();
public int getDataVersion();
public void setDataVersion(int nVersion);
public Binary getFutureData();
public void setFutureData(Binary binFuture);
}

The Portable Object Format
(advanced serialization)

© 2009 Oracle Corporation
The Portable Object Format
(advanced serialization)



Steps to use Evolvable Portable Objects (in Java)
1.

Implement
com.tangosol.io.pof.EvolvablePortableObject

2.

Consider extending
com.tangosol.io.AbstractEvolvable
3.

Maintain version number (update when new fields are
added)
4.

That’s it!

public class Position extends
AbstractEvolvable

implements
EvolvablePortableObject
{

...
public int getImplVersion() {
return VERSION;
}


© 2009 Oracle Corporation
The Portable Object Format
(advanced serialization)



Evolvable: The Fine Print


When an older class deserializes data from a newer class
version, it holds on to the “remainder” (this field is called
FutureData)


Upon serialization, this FutureData is written back to the
stream, thus preserving the data written by the newer version
of the class


This means that new fields can be
added
, but they cannot be
changed to a different type
or
deleted!

© 2009 Oracle Corporation
The Portable Object Format
(advanced serialization)



Custom Serializer


Implementing
PortableObject
is not the only way to
serialize objects using POF



A
PofSerializer
can be coded/configured to perform
serialization

public interface PofSerializer
{
public void serialize(PofWriter out, Object o)
throws IOException;
public Object deserialize(PofReader in)
throws IOException;
}

© 2009 Oracle Corporation
POF Example
PositionSerializer.java

public class PositionSerializer
implements PofSerializer
{
public void serialize(PofWriter out, Object o) throws IOException {
Position p = (Position)o;

out.writeInt(0, p.getId());
out.writeString(1, p.getSymbol());


// etc...


out.writeRemainder(null);
}
public Object deserialize(PofReader in) throws IOException {
Position p = new Position();

int id = in.readInt(0);
String symbol = in.readString(1);

// etc...


in.readRemainder();

return new Position(id, symbol, ...);
}
}
© 2009 Oracle Corporation
POF Example
pof-config.xml
(including serializer)

<pof-config>


<user-type-list>




<!-- include all of the standard Coherence type -->
<include>coherence-pof-config.xml</include>




<!-- our custom types (user types) -->



<user-type>



<type-id>1000</type-id>


<class-name>Position</class-name>
<serializer>
<class-name>PositionSerializer</class-name>
</serializer>



</user-type>
</user-type-list>



</pof-config>


© 2009 Oracle Corporation
The Portable Object Format
(advanced serialization)



POF Extractor


The
ValueExtractor
interface defines a method to extract
a value from an object



For example:
new ReflectionExtrator(“getSymbol”)



In general this is used for executing queries and aggregations
in the grid


It does require deserializing the object in order to execute the
method defined in the extractor

© 2009 Oracle Corporation
The Portable Object Format
(advanced serialization)



POF Extractor


However, using
POFExtractor
, instead of deserializing the
object, the value can be extracted directly from the POF
binary by specifying the index:



For example:
new PofExtractor(2)



For large objects that are expensive to deserialize, this will
provide huge savings in terms of CPU to execute an extractor
© 2009 Oracle Corporation
The Portable Object Format
(advanced serialization)



New in Coherence 3.5


POF Extractor (described in previous section)



New features for easier development/debugging



Portable Object no longer implements Serializable


Makes it harder to accidentally use Java serialization



Serialization stack traces are more meaningful


Easier to track down (de)serialization bugs


POF specification will be published

© 2009 Oracle Corporation
The Portable Object Format
Conclusion



POF has “evolved” to become the preferred
serialization format for Coherence


Starting in 3.4



All future serialization enhancements will be
based on POF


Q/A
© 2009 Oracle Corporation
For More Information
search.oracle.com
Or
oracle.com/technology/products/coherence/index.html
Coherence
© 2009 Oracle Corporation
The preceding is intended to outline our general
product direction. It is intended for information
purposes only, and may not be incorporated into any
contract. It is not a commitment to deliver any
material, code, or functionality, and should not be
relied upon in making purchasing decisions.
The development, release, and timing of any
features or functionality described for Oracle’s
products remains at the sole discretion of Oracle.
© 2009 Oracle Corporation
© 2009 Oracle Corporation