[Zope-CVS] CVS: Products/Ape/lib/apelib/zodb3 - serializers.py:1.3storage.py:1.8

[Shane Hathaway]

On Tue, 16 Sep 2003, Tim Peters wrote:

> [Shane Hathaway]
> > Modified Files:
> > 	serializers.py storage.py
> > Log Message:
> > Fixed Python 2.3 compatibility.
> >
> > - If you append to a list while it is being pickled, Python 2.3 will
> >   pick up the new items.  Previous versions of Python did not.
> 
> How do you grow a list while it's being pickled?  Must be another thread
> doing that, and if so that's mighty dubious.  If you're pickling to a
> genuine file object, then cPickle releases the GIL around its fwrite()
> calls, so that's one way.  But in the context of the patch:
> 
> +        p.persistent_id = lambda ob: None  # Stop recording references
>          p.dump(unmanaged)
>          s = outfile.getvalue()
>          event.addUnmanagedPersistentObjects(unmanaged)

Actually, the persistent_id() function was appending to the "unmanaged" 
list on each call.  Here is a simplified version of the mistake:


unmanaged = []

def persistent_id(ob):
    unmanaged.append(ob)
    return None

s = StringIO()
p = Pickler(s)
p.persistent_id = persistent_id
p.dump(unmanaged)


Python 2.2 apparently records the length of the list being pickled before
analyzing its contents.  This meant that my mistake went unnoticed.  (I
did not intend to modify "unmanaged" while it was being pickled!  There
are two passes in the actual code; the "unmanaged" list is built up in the
first pass and pickled in the second pass.)  Python 2.3, on the other
hand, picks up each new item as it is appended, resulting in a finite but
very long loop.

> Copying Python-Dev because this may reveal a new mountain of ways to crash
> the interpreter:  mutate objects in devious ways while they're getting
> cPickled.

I wouldn't be too concerned.  Only people who do crazy things with 
persistent_id() hooks will be affected. ;-)

Shane