[Zope-dev] ZODB: interface for walking object graph?

[Michel Pelletier]

"Andrew M. Kuchling" wrote:
> 
> I'd like a way to find all instances of a given class in a ZODB.  (In
> ODBMS jargon, this feature seems to be called "class extents".)  The
> purpose is to handle changes to a class that require rearranging; you
> can do it lazily in a __setstate__ method, but after you've had 10
> changes, that __setstate__ will be really complicated. 

It's not too bad, although you are correct that allways get more complex
and never less.  We have generally been able to avoid incredibly complex
setstates because we try to not evolve an object to incredibly, as
opposed to engineering a new kind of object that abstracts out or
delegates or whatever.

Also, if you just want to add an attribute you can add a shared instance
attribute to the class definition.  We do this alot to avoid changing
setstate just to add an attribute.  This leaves much of our setstating
to doing some kind of transformation.

> And you can
> never simplify it, because you never know if there's a version 1
> object hiding somewhere that just hasn't been accessed yet.  Instead,
> I'd like to do it eagerly; write a script that loops over all the
> instances of class C, make a change to all of them, and then commit.

I think this could cause collisions and possibly aborts if your objects
are being changed actively.
 
> I only see one way of doing this, by walking over the whole object
> graph starting from the ZODB's root objects.  Storage objects do this
> in their pack() method to determine reachability. 

But they don't actually change anything, I don't think, thus no worries
of collisions and no transaction commited.

> Problem: you'd have
> to copy the walking logic out of the pack() method into your own
> method.
> 
> Proposal: expose some interface allowing clients of a Connection
> to run a function over every single object in a ZODB.  It'd be like
> os.path.walk(), except over objects, not directories and files.
> 
> This means unpickling *every* *single* object in the ZODB (and sending
> them all over the wire if you're using ZEO). 

(unless they're cached, but if you change them it gets worse,
invalidation messages galore)

> Inefficient, but there
> seems no other alternative, and you wouldn't do this very often.

Which makes me thing you would want to do something like this analgously
to UNIX 'single user mode', ie, shut down your site, and run a single
thread to execute this method of yours (immagine, for example, what
would happen if two threads decided to run this method of yours at the
same time).  It's pretty drastic, in 9 out of 10 cases you could
probably just get away with setstate.

-Michel