[Zope] persistence and dictionaries - new light

Matt matt.bion@eudoramail.com
Sat, 09 Dec 2000 16:50:32 +1300


I tried a few things out it seems the following is happening :

the methods that seemed to be activating persistence when I thought they
shouldn't are called through my handle to the manage_edit ... i.e. in the
product I sent this was anything called within

 def manage_edit(self, title, REQUEST=None):
    blah

If I call the same methods from another method in the product, and then call
that method from say a dtmlMethod then persotence failed unless I use
_p_changed.  This would go along with what Chris mentioned.

So it seems that def manage_edit has a special behaviour, possible linked to
it's binding to
manage_main = HTMLFile('boringEdit', globals()) # Management Interface
which is linked to
manage_options = (
        {'label': 'Edit',       'action': 'manage_main'},

hopefully someone has an answer to that.

regards
Matt



Jonothan Farr wrote:

> Oops. You snipped the contents of MoreStuff.addStuff(). No way to tell what's
> going on without that!
> --jfarr
>
> ----- Original Message -----
> From: "Matt" <matt.bion@eudoramail.com>
> To: "Jonothan Farr" <jfarr@real.com>; <zope@zope.org>
> Sent: Friday, December 08, 2000 2:39 PM
> Subject: Re: [Zope] persistence and dictionaries
>
> > Ok, here are some of the offending bits out of my boringplus product
> > .... which I can send again if people want.  I have
> > another product that keeps a dictionary of cookies so I can see how many
> > times someone uses the back button to
> > access the same forms page again.  This worked as expected, where I had
> > to issue an _p_changed=1 to get them to
> > persist.  But I am doing a similar thing below, or so I thought, once
> > with a dictionary in the product and once in a
> > dictionary within a class that does not inherit Persistent.  This is
> > what is confusing.  Through restarts etc, everything
> > perisists
> >
> >
> > class Boring(
> >     OFS.SimpleItem.Item,   # A simple Principia object. Not Folderish.
> >     Persistent,            # Make us persistent. Yaah!
> >     Acquisition.Implicit,  # Uh, whatever.
> >     AccessControl.Role.RoleManager # Security manager.
> >     ):
> >
> >
> > def __init__(self, id, title=''):
> >      self._things = {}
> >      self._more = myClass.MoreStuff()
> >
> >
> > def manage_edit(self, title, REQUEST=None):
> >    if REQUEST is not None:
> >       if REQUEST.has_key('thing'):
> >          self._things[REQUEST['thing']] = REQUEST['thing_value']
> >   self.addToMore(REQUEST['thing'],REQUEST['thing_value'])
> >
> >
> > def addToMore(self,name,value):
> >         self._more.addStuff(name,value)
> >         return ""
> >
> >
> > class MoreStuff:
> >
> >     def __init__(self):
> >         self._stuff = {'animal':'monkey'}
> >
> >     def addStuff(self,stuff_name,stuff_item):
> >
> >
> > On Sat, 09 Dec 2000, Jonothan Farr wrote:
> > > Maybe I'm mistaken, but it seems like you can put an instance of an
> > object that
> > > doesn't inherit from Persistent into the ZODB just fine, but its
> > contents won't
> > > persist, so you'll always end up with a copy of the object as it was
> > first added
> > > to the database.
> > >
> > > --jfarr
> > >
> > > ----- Original Message -----
> > > From: "Chris McDonough" <chrism@digicool.com>
> > > To: <matt.bion@eudoramail.com>
> > > Cc: <zope@zope.org>
> > > Sent: Friday, December 08, 2000 11:57 AM
> > > Subject: Re: [Zope] persistence and dictionaries
> > >
> > >
> > > > Huh.  If they do, it's by chance only.  I'd be hard-pressed to
> > explain it.
> > > >
> > > > Do they inherit from *anything*?
> > > >
> > > > ----- Original Message -----
> > > > From: "Matt" <matt.bion@eudoramail.com>
> > > > To: "Chris McDonough" <chrism@digicool.com>
> > > > Cc: <zope@zope.org>
> > > > Sent: Friday, December 08, 2000 2:30 PM
> > > > Subject: Re: [Zope] persistence and dictionaries
> > > >
> > > >
> > > > > Chris, this was my original confusion .... the two places below
> > where you
> > > > say
> > > > >
> > > > > "You can put instances which do not inherit from
> > Persistence.Persistent in
> > > > > your database.  They just won't "stick".  They'll hang around
> > until the
> > > > > server is restarted or for an undetermined amount of time during
> > normal
> > > > > operations."
> > > > >
> > > > > "No.  It'll work for "a while" or until the server is restarted.
> > :-)"
> > > > >
> > > > > actually do persist after restarts ... that's what confused me,
> > they
> > > > wouldn't
> > > > > go away and they should!!
> > > > >
> > > > > regards
> > > > > Matt
> > > > >
> > > > > Chris McDonough wrote:
> > > > >
> > > > > > > Thanks for the reply, that is really useful.  There are a
> > couple of
> > > > things
> > > > > > > though that still don't add up.  Firstly, you say below, as do
> > all the
> > > > > > ZODB
> > > > > > > documents that "Custom" classes can certainly persist, they
> > just need
> > > > to
> > > > > > mix in
> > > > > > > the "Persistence.Persistent" class as a base class.  Well, in
> > my
> > > > example I
> > > > > > > attached in my first email, my product certainly has
> > > > > > Persistence.Persistent,
> > > > > > > but my second class that I add to this one does not, yet it
> > still
> > > > > > persists.
> > > > > > > There was an email sometime ago on the mailing list that told
> > someone
> > > > that
> > > > > > this
> > > > > > > was why their product instances disappearing from the ZODB.
> > > > > > > (the ref for the original email is :
> > > > > > http://www.egroups.com/message/zope/44263
> > > > > > > ... I can't find the reply again.)
> > > > > > >
> > > > > >
> > > > > > You can put instances which do not inherit from
> > Persistence.Persistent
> > > > in
> > > > > > your database.  They just won't "stick".  They'll hang around
> > until the
> > > > > > server is restarted or for an undetermined amount of time during
> > normal
> > > > > > operations.
> > > > > >
> > > > > > > So my current understanding would be that any classes you want
> > to add
> > > > in
> > > > > > do not
> > > > > > > need to derive from Persistence.Persistent, and if it is
> > pickleable
> > > > then
> > > > > > all
> > > > > > > should be fine if you call on instances of that object within
> > you
> > > > product.
> > > > > >
> > > > > > No.  It'll work for "a while" or until the server is restarted.
> > :-)
> > > > > >
> > > > > > > The next part that worried me came from the "python product
> > tutorial"
> > > > > > > http://www.zope.org/Members/hathawsh/PythonProductTutorial
> > > > > > >
> > > > > > > This stated that the class dictionary self.votes = {} needed
> > to be
> > > > changed
> > > > > > to
> > > > > > > self._votes = Globals.PersistentMapping()  so that updates to
> > it
> > > > persist.
> > > > > > > Hence my query about dictionaries.
> > > > > >
> > > > > > This was for convenience, I'd imagine.
> > > > > >
> > > > > > >  I also noticed your comment about __setstate__ .  What is it
> > about
> > > > this
> > > > > > that is
> > > > > > > dangerous.
> > > > > >
> > > > > > Nothing implicitly dangerous, but it can get confusing if you
> > have
> > > > multiple
> > > > > > revisions of your product and you use variables caused by
> > __setstate__.
> > > > > > Also, once you add a __setstate__ which modifies the object
> > in-place,
> > > > > > there's a likelihood that it can never go away (you're can never
> > be sure
> > > > if
> > > > > > all instances have been updated).
> > > > > >
> > > > > > > Recently I built a product out of some python classes I
> > wrapped
> > > > > > > around 4DOM, and since 4DOM documents do not seem to
> > persist(well the
> > > > > > document
> > > > > > > does, but it loses all its children), then I persisted them to
> > the
> > > > local
> > > > > > file
> > > > > > > system, since I needed to do that anyway for what I was doing.
> >
> > > > Setstate
> > > > > > seemed
> > > > > > > to work nicely to bring them back, though watching its
> > behaviour I
> > > > noticed
> > > > > > that
> > > > > > > it was called very often by zope.
> > > > > >
> > > > > > Sure, that works... although at that point you're creating your
> > own
> > > > object
> > > > > > database.  :-)
> > > > > >
> > > > > > >
> > > > > > > Chris McDonough wrote:
> > > > > > >
> > > > > > > > All pickleable Python primitive types (strings,
> > dictionaries, lists,
> > > > > > Nones,
> > > > > > > > integers, floats, longs, etc.) can live in the ZODB.  They
> > can
> > > > persist
> > > > > > just
> > > > > > > > like instances that inherit from the Persistent class.
> > > > > > > >
> > > > > > > > I think you read a little too much in to the fact that you
> > need to
> > > > > > "treat
> > > > > > > > mutable objects immutably" when working with them in the
> > ZODB.  This
> > > > > > > > statement doesn't mean that these kinds of objects can't be
> > saved in
> > > > the
> > > > > > > > ZODB, it just means you need to treat them specially when
> > putting
> > > > them
> > > > > > in
> > > > > > > > the database.
> > > > > > > >
> > > > > > > > For instance, if you were doing this inside of an external
> > method:
> > > > > > > >
> > > > > > > > def amethod(self):
> > > > > > > >    self.mydict = {}
> > > > > > > >    self.mydict['a'] = 1
> > > > > > > >
> > > > > > > > (where self is the persistent object that is usually the
> > external
> > > > > > method's
> > > > > > > > "container")
> > > > > > > >
> > > > > > > > It wouldn't work as you expected.  Although you'd see an 'a'
> > in
> > > > mydict
> > > > > > for a
> > > > > > > > little while in further accesses to it, 'mydict' would
> > eventaully
> > > > show
> > > > > > up as
> > > > > > > > an empty dictionary on the first access of it after it was
> > expired
> > > > from
> > > > > > the
> > > > > > > > RAM cache (after it was 'ghosted'), because the last thing
> > that the
> > > > ZODB
> > > > > > > > "saw" (via the __setattr__ on 'self' and a subsequent
> > transaction)
> > > > was
> > > > > > you
> > > > > > > > setting a empty dictionary.
> > > > > > > >
> > > > > > > > Persistent objects (like "self" in the above example) are
> > only smart
> > > > > > enough
> > > > > > > > to notice changes to themselves that happen through their
> > > > __setattr__
> > > > > > (e.g.
> > > > > > > > self.mydict = {} calls self's __setattr__).  Mutating the
> > attribute
> > > > > > 'mydict'
> > > > > > > > above "in-place" (via self.mydict['a'] = 1) does not trigger
> > self's
> > > > > > > > __setattr__, so the ZODB never notices that "mydict" got
> > changed.
> > > > > > > >
> > > > > > > > There are two ways to handle this.  The first is to treat
> > mutable
> > > > > > attributes
> > > > > > > > "immutably" via assigning to a temporary variable and then
> > making
> > > > sure
> > > > > > the
> > > > > > > > persistent container's __setattr__ gets called:
> > > > > > > >
> > > > > > > > def amethod(self):
> > > > > > > >    dict = {}
> > > > > > > >    dict['a'] = 1
> > > > > > > >    self.mydict = dict # trigger persistence mechanism
> > implicitly
> > > > > > > >
> > > > > > > > The second is to use the _p_changed attribute of the
> > persistent
> > > > object
> > > > > > on
> > > > > > > > which the primitive is set.  This explcitly tells the
> > persistence
> > > > system
> > > > > > to
> > > > > > > > include the object on which it's set into the current
> > transaction:
> > > > > > > >
> > > > > > > > def amethod(self):
> > > > > > > >    self.mydict = {}
> > > > > > > >    self.mydict['a'] = 1
> > > > > > > >    self._p_changed = 1 # trigger persistence mechanism
> > manually
> > > > > > > >
> > > > > > > > Variations on this theme extend to list methods too (e.g.
> > > > list.append,
> > > > > > > > list.pop, etc.)
> > > > > > > >
> > > > > > > > "Custom" classes can certainly persist, they just need to
> > mix in the
> > > > > > > > "Persistence.Persistent" class as a base class.
> > > > > > > >
> > > > > > > > As long as you obey this (slightly annoying, but necessary)
> > rule,
> > > > you
> > > > > > > > shouldn't need to use PersistentMapping (it's really just a
> > > > convenient
> > > > > > > > wrapper that does this "magic" for you) or make any other
> > wrappers
> > > > for
> > > > > > other
> > > > > > > > mutable objects.
> > > > > > > >
> > > > > > > > I don't know why you're using __setstate__, but ummmm.. I
> > won't go
> > > > > > there.
> > > > > > > > :-)  I didn't look at your product, but I don't think I need
> > to to
> > > > > > answer
> > > > > > > > your question...
> > > > > > > >
> > > > > > > > > Hi I am trying to get a handle on how I should handle
> > peristence
> > > > in my
> > > > > > > > > python products.  I have read all the ZODB documents and
> > all the
> > > > > > Product
> > > > > > > > > tutorials, which all led me to believe that 1) mutable
> > objects
> > > > such as
> > > > > > > > > lists and dictionaries are not persistent and that updates
> > to
> > > > these
> > > > > > will
> > > > > > > > > not be implicitly saved into the ZODB, and 2) that custom
> > classes
> > > > > > would
> > > > > > > > > certainly not persist.  That all seemed fine, and I used
> > persitent
> > > > > > > > > mapping and __setstate__ to fill things back in where
> > necessary.
> > > > But
> > > > > > > > > then I decided to demonstrate that persitence does break
> > as
> > > > suggested.
> > > > > > > > >
> > > > > > > > > I used boring product, added a dicitonary and a custom
> > class that
> > > > > > > > > contains it's own dictionary.... let the user update name
> > : value
> > > > > > pairs
> > > > > > > > > for both, and print the contents through index.dtml
> > > > > > > > >
> > > > > > > > > The problem is that everything persists fine !!!!  through
> >
> > > > restarts,
> > > > > > > > > everything?
> > > > > > > > >
> > > > > > > > > Why does it work???  shouldn't it not?
> > > > > > > > >
> > > > > > > > > I have attached the modified boring product ....
> > boringplus .....
> > > > it's
> > > > > > > > > dead simple to follow if you have made products before.
> > > > > > > > >
> > > > > > > > > Any explanation would be really nice.
> > > > > > > > >
> > > > > > > > > regards
> > > > > > > > > Matt
> > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > > >
> > > > >
> > > > >
> > > > > _______________________________________________
> > > > > Zope maillist  -  Zope@zope.org
> > > > > http://lists.zope.org/mailman/listinfo/zope
> > > > > **   No cross posts or HTML encoding!  **
> > > > > (Related lists -
> > > > >  http://lists.zope.org/mailman/listinfo/zope-announce
> > > > >  http://lists.zope.org/mailman/listinfo/zope-dev )
> > > > >
> > > > >
> > > >
> > > >
> > > > _______________________________________________
> > > > Zope maillist  -  Zope@zope.org
> > > > http://lists.zope.org/mailman/listinfo/zope
> > > > **   No cross posts or HTML encoding!  **
> > > > (Related lists -
> > > >  http://lists.zope.org/mailman/listinfo/zope-announce
> > > >  http://lists.zope.org/mailman/listinfo/zope-dev )
> > > >
> > >
> > >
> > > _______________________________________________
> > > Zope maillist  -  Zope@zope.org
> > > http://lists.zope.org/mailman/listinfo/zope
> > > **   No cross posts or HTML encoding!  **
> > > (Related lists -
> > >  http://lists.zope.org/mailman/listinfo/zope-announce
> > >  http://lists.zope.org/mailman/listinfo/zope-dev )
> >
> >
> > _______________________________________________
> > Zope maillist  -  Zope@zope.org
> > http://lists.zope.org/mailman/listinfo/zope
> > **   No cross posts or HTML encoding!  **
> > (Related lists -
> >  http://lists.zope.org/mailman/listinfo/zope-announce
> >  http://lists.zope.org/mailman/listinfo/zope-dev )
> >
>
> _______________________________________________
> Zope maillist  -  Zope@zope.org
> http://lists.zope.org/mailman/listinfo/zope
> **   No cross posts or HTML encoding!  **
> (Related lists -
>  http://lists.zope.org/mailman/listinfo/zope-announce
>  http://lists.zope.org/mailman/listinfo/zope-dev )