Supporting Persistent Objects

hedgebornabaloneΛογισμικό & κατασκευή λογ/κού

2 Δεκ 2013 (πριν από 4 χρόνια και 5 μήνες)

82 εμφανίσεις

Supporting Persistent Objects

In Python

Jeremy Hylton

(C) 2002 Zope Corp.

What is persistence?

Data lives longer than programs

Files, pipes, relational databases, etc.

But, discontinuity of representation

Orthogonal persistence

Automatic management of program state

Independent of data type or longevity

Allow programmer to focus on application
data model

(C) 2002 Zope Corp.

Python approach


Minimize changes to existing programs

Work with standard interpreter

Zope Object Database (ZODB)

Support Zope application server

Originally targeted for web development

Separate database and storage layers

(C) 2002 Zope Corp.

Not on today’s agenda

Benefits of persistence

Discussion of related work

Early work

algol, Napier88

Much recent work on Java

PJava, ObjectStore PSE

(C) 2002 Zope Corp.

ZODB key ideas

Persistence by reachability

Any reachable object is saved

Transactions to control updates

Safe sharing among clients

Implement with Persistent mixin

Provides hook for persistence

Compromise on transparency

(C) 2002 Zope Corp.

Persistent base class

C extension type that defines

Four _p_ attributes

Custom attribute accessors

Efficient C API

Marks serialization boundaries

Interacts with database

Load objects on demand

Register modifications with TM

(C) 2002 Zope Corp.

Persistence by reachability

Any object reachable from ZODB root
is persistent

(C) 2002 Zope Corp.

A simple example

from Persistence import Persistent

from Transaction import get_transaction

from ZODB.FileStorage import DB

class Counter(Persistent):

_value = 0

def inc(self):

self._value += 1

def main():

fs = DB(“data.fs”)

conn =; root = conn.root()

obj = root[“myobj”] = Counter()



(C) 2002 Zope Corp.

Object serialization

Standard pickle library

Serializes arbitrary object graph

Raises TypeError for sockets, files, &c.

Instance vars serialized via dictionary

Hooks to define custom state

__getstate__() / __setstate__()

Persistent mixin ignores _v_ attributes

(C) 2002 Zope Corp.

Pickling persistent objects

Stores objects in separate records

Persistent objs pickled as oid + class

Works with cache to maintain identity

Handling non
persistent objects

Copied into record of containing object

Sharing by persistent objs is problematic

(C) 2002 Zope Corp.

Object identity / caching

Cache maintains oid

obj mapping

Guarantees only one copy of object

Unpickler loads all referenced objects

Ghost objects

Only Persistent header initialized

No instance state loaded

State loaded on first object access

LRU cache of recent objects

(C) 2002 Zope Corp.

Attribute access handlers

Persistent implements C wrappers

Override tp_getattro, tp_setattro slots

Mediate access to instance variables

Crucial Python feature

(C) 2002 Zope Corp.

Example __getattribute__() hook

class Example(object):

_p_state = False

def _p_activate(self):

print "activate"

self._p_state = True

def __getattribute__(self, attr):

print "intercept", attr

if not attr.startswith("_p_") and not self._p_state:


return super(Example, self).__getattribute__(attr)

>>> obj = Example(); obj.value = "test"

>>> print obj.value

intercept value

intercept _p_state

intercept _p_activate



(C) 2002 Zope Corp.


Supports multiple threads, processes

Independent database connections

Updates visible at transaction boundaries

Optimistic concurrency control

When conflict occurs, abort and retry

On error, abort to restore consistency

Reverts to last saved state

(C) 2002 Zope Corp.

Concurrency and conflicts

Invalidations sent at commit time

Clients process at transaction boundaries

Conflicting transactions aborted

Write conflict at commit time

Read conflict on object access

Application must retry on conflict

Can use generic wrapper

Can define conflict resolution method

(C) 2002 Zope Corp.

Retrying conflicts

Example wrapper for retries

def transact(f, retries=3):

def wrapper(*args, **kwargs):

n = retries

while n:



return f(*args, **kwargs)



except ConflictError:

= 1

if n == 0:





return wrapper

(C) 2002 Zope Corp.

Other features

Undo support

Storage stores multiple revisions

Transactional undo reverts to earlier state

BTrees: efficient persistent containers

Storing code in database

(C) 2002 Zope Corp.


Schema evolution

Must code manually in __setstate__()

Database management

Manual pack() to remove revisions, do GC

Sharing of non
persistent objects

Integration with legacy code

Multiple inheritance helps

Factory classes

(C) 2002 Zope Corp.

Getting the software

info central for ZODB

ZODB 3.1 released Oct. 2002

Uses ExtensionClass

ZODB4 alpha planned for Dec. 2002

Based on Python 2.2 type support

Fewer onions in the varnish