Supporting Persistent Objects

hedgebornabaloneΛογισμικό & κατασκευή λογ/κού

2 Δεκ 2013 (πριν από 3 χρόνια και 8 μήνες)

68 εμφανίσεις

Supporting Persistent Objects

In Python



Jeremy Hylton

jeremy@zope.com

(C) 2002 Zope Corp.

What is persistence?


Data lives longer than programs


Files, pipes, relational databases, etc.


But, discontinuity of representation


Orthogonal persistence


Automatic management of program state


Independent of data type or longevity


Allow programmer to focus on application
data model

(C) 2002 Zope Corp.

Python approach


Goals


Minimize changes to existing programs


Work with standard interpreter


Zope Object Database (ZODB)


Support Zope application server


Originally targeted for web development


Separate database and storage layers

(C) 2002 Zope Corp.

Not on today’s agenda


Benefits of persistence


Discussion of related work


Early work


PS
-
algol, Napier88


Much recent work on Java


PJava, ObjectStore PSE

(C) 2002 Zope Corp.

ZODB key ideas


Persistence by reachability


Any reachable object is saved


Transactions to control updates


Safe sharing among clients


Implement with Persistent mixin


Provides hook for persistence


Compromise on transparency

(C) 2002 Zope Corp.

Persistent base class


C extension type that defines


Four _p_ attributes


Custom attribute accessors


Efficient C API


Marks serialization boundaries


Interacts with database


Load objects on demand


Register modifications with TM

(C) 2002 Zope Corp.

Persistence by reachability


Any object reachable from ZODB root
is persistent


(C) 2002 Zope Corp.

A simple example


from Persistence import Persistent

from Transaction import get_transaction

from ZODB.FileStorage import DB


class Counter(Persistent):


_value = 0


def inc(self):



self._value += 1


def main():


fs = DB(“data.fs”)


conn = db.open(); root = conn.root()


obj = root[“myobj”] = Counter()


get_transaction().commit()


obj.inc()


get_transaction().commit()

(C) 2002 Zope Corp.

Object serialization


Standard pickle library


Serializes arbitrary object graph


Raises TypeError for sockets, files, &c.


Instance vars serialized via dictionary


Hooks to define custom state


__getstate__() / __setstate__()


Persistent mixin ignores _v_ attributes

(C) 2002 Zope Corp.

Pickling persistent objects


Stores objects in separate records


Persistent objs pickled as oid + class


Works with cache to maintain identity


Handling non
-
persistent objects


Copied into record of containing object


Sharing by persistent objs is problematic

(C) 2002 Zope Corp.

Object identity / caching


Cache maintains oid


obj mapping


Guarantees only one copy of object


Unpickler loads all referenced objects


Ghost objects


Only Persistent header initialized


No instance state loaded


State loaded on first object access


LRU cache of recent objects

(C) 2002 Zope Corp.

Attribute access handlers


Persistent implements C wrappers


Override tp_getattro, tp_setattro slots


Mediate access to instance variables


Crucial Python feature

(C) 2002 Zope Corp.

Example __getattribute__() hook

class Example(object):


_p_state = False


def _p_activate(self):


print "activate"


self._p_state = True


def __getattribute__(self, attr):


print "intercept", attr


if not attr.startswith("_p_") and not self._p_state:


self._p_activate()


return super(Example, self).__getattribute__(attr)


>>> obj = Example(); obj.value = "test"

>>> print obj.value

intercept value

intercept _p_state

intercept _p_activate

activate

test

(C) 2002 Zope Corp.

Transactions


Supports multiple threads, processes


Independent database connections


Updates visible at transaction boundaries


Optimistic concurrency control


When conflict occurs, abort and retry


On error, abort to restore consistency


Reverts to last saved state

(C) 2002 Zope Corp.

Concurrency and conflicts


Invalidations sent at commit time


Clients process at transaction boundaries


Conflicting transactions aborted


Write conflict at commit time


Read conflict on object access


Application must retry on conflict


Can use generic wrapper


Can define conflict resolution method

(C) 2002 Zope Corp.

Retrying conflicts


Example wrapper for retries

def transact(f, retries=3):


def wrapper(*args, **kwargs):


n = retries


while n:


try:


try:


return f(*args, **kwargs)


finally:


get_transaction().commit()


except ConflictError:


n
-
= 1


if n == 0:


raise


except:


get_transaction().abort()


raise


return wrapper

(C) 2002 Zope Corp.

Other features


Undo support


Storage stores multiple revisions


Transactional undo reverts to earlier state


BTrees: efficient persistent containers


Storing code in database

(C) 2002 Zope Corp.

Limitations


Schema evolution


Must code manually in __setstate__()


Database management


Manual pack() to remove revisions, do GC


Sharing of non
-
persistent objects


Integration with legacy code


Multiple inheritance helps


Factory classes

(C) 2002 Zope Corp.

Getting the software


http://www.zope.org/Wikis/ZODB


info central for ZODB


ZODB 3.1 released Oct. 2002


Uses ExtensionClass


ZODB4 alpha planned for Dec. 2002


Based on Python 2.2 type support


Fewer onions in the varnish