[Zope-Coders] ZRDB and Transactions

[Jim Fulton]

I wish I had time to look at your code, but I don't. This isn't
because I think what your doing is unimportant, I just have too
many commitments. I will try to help as much as I can.

Here are some thoughts.

- We *really* need to do a better job of documenting the
  Transaction framework!  Specifically wrt two-phase commit.

- It's important that RDBMS transactions not get committed
  till the second phase of the two-phase commit (i.e. tpc_finish).
  Conflicts are detected during the first phase, so defering
  RDBMS commits till the second phase should avoid the problem
  you are describing.  

  (Note however, that second-phase operaions
  are not allowed to fail. Ideally, we should be integrating with
  the RDBMS' TP-monitor/two-phase commit APIs, but we don't currently
  have Python interfaces to those. :/)

If these observations (especially the second ;) don't
reveal an answer, I'll try to help, but if it comes to that, please
try to simplify things for me as much as possible.

Jim

Stephan Richter wrote:
> 
> Hello everyone,
> 
> since this is a real low-level core issue and I am looking for the
> maintainer (well, I am to 90% sure that it is Jim ;-) ), I am going to post
> it here, instead of Zope-Dev:
> 
> The problem revolves around DBObjects, even though it is not particular to
> it. As you may know, when a DBObject is created, a SQL Method is called
> saving the data. This method also returns an id that is then used as the
> Zope object's id as well.
> 
> This all works fine until we get a ConflictError while writing the
> Zope-Fingerprint of the DBObject. Of course, the Publisher then aborts the
> publishing process and creates a new request to retry the publishing
> process (ZPublisher/Publish.py; line 175-205). This process then passes and
> everything seems fine. It created as many Zope Fingerprints as it did
> Database entries.
> 
> But when l look at the entries itself, I notice that the ids are out of
> sync. After some testing I noticed that PostGreSQL saved the data (and
> therefore id) of the first attempt and the Zope fingerprint (ZODB) saved
> the id of the second attempt. I tested the DA and it does not do anything
> weird.
> 
> I tested this behavior primarily with Zope 2.3.3 and Python 1.5.2, since
> PsycoPG is not yet stable for Python 2.x (throwing segmentation faults). I
> also tested this configuration with PoPy and its Zope DA.
> 
> Since PoPy runs stable under Python 2.x (except a memory leak, which exists
> on all Python versions), I tried it there again as well. I used Zope
> 2.4.2b1. The same problem exists. So there is definitely no fix in the
> latest Zope version.
> 
> Even though I could not find any clues, I think something might be wrong in
> the way the Transactions are committed (ZODB/Transaction.py; line 258-383).
> I noticed that the "Surrogate" object of the DB DA is always first in the
> commit list. But when I reversed the order, it would not change the
> situation. So, I am really lost on what to try next.
> 
> If you like to receive the Test Suite I did all the tests with, please feel
> free to drop me a mail. I would not even mind giving a couple people
> (people I know) access to my server in order to run the tests locally.
> 
> DBObjects URL: http://demo.iuveno-net.de/iuveno/Products/DBObjects
> 
> Regards,
> Stephan
> 
> --
> Stephan Richter
> CBU - Physics and Chemistry Student
> Web2k - Web Design/Development & Technical Project Management
> 
> _______________________________________________
> Zope-Coders mailing list
> Zope-Coders@zope.org
> http://lists.zope.org/mailman/listinfo/zope-coders

--
Jim Fulton           mailto:jim@zope.com       Python Powered!        
CTO                  (888) 344-4332            http://www.python.org  
Zope Corporation     http://www.zope.com       http://www.zope.org