[Zope-DB] Zope, DCOracle2, concurrent transactions, etc.

Matthew T. Kromer matt at bane.mi.org
Thu May 4 07:44:26 EDT 2006

I'm pretty confident that the db component (the C layer) doesn't have  
any GIL problems.  However, the DA component (the Zope layer) *may*  
have all sorts of unusual failure modes.

Debugging transaction concurrency problems is always tricky, but keep  
in mind there are Python layer abstractions that get and release  
locks as well.

I'm speculating based on the history of your discussion thread that  
you're running out of Zope db connections.  The way it works in Zope  
is somewhat unusual; there are usually NOT enough object database  
connections if you run more than four threads (unless someone fixed  
this way back when, the default was 7).  Threads in Zope wait in line  
for a zodb connection to become available.  Under normal  
circumstances they don't have to wait, but in yours I think they are  
waiting.  Are you trying to run with more than four threads?  If so  
you might want to tweak the pool_size parameter up in the DB class of  
lib/python/ZODB/DB.py.  There are negative consequences of doing this  
(ie each DB connection cache gobbles memory) but on the upside you  
might be able to avoid stalling out Zope.

This won't provide you relief the way you might hope; if the problem  
is that threads are entering a long running transaction and waiting  
on Oracle to complete, all you're doing is adding a few more feet to  
the rope before Zope hangs itself up again.

DCOracle2 does support event tracing, so you could turn on the trace  
log to see what the module is doing when Zope is becoming  
nonresponsive.  It probably will say that it's waiting in a fetch or  
execute statement, so that won't be all too terribly helpful but it  
MIGHT indicate that there's a problem with handles (some versions of  
DCOracle2 leaked a handle -- make sure you have the latest code from  
the CVS repository).

Also, the C layer isn't capable of coping with some result types very  
well (if at all) ie using Oracle in object mode or returning table  
results from stored procedures, etc.

Also, the PYTHON layer code that currently exists as the DA *does*  
leak connections if Oracle closes the connection.  This is something  
Chris Withers tried to fix, but I think he was never happy with the  
solution and so it remains on a branch.

I haven't done any DCO2 development in a little over two years; I  
hate to abandon the code but I don't run any current Oracles at home  
and its a fairly intensive time commitment to just set up a  
development environment.  Oracle never gave me any freebies either, / 
sniff -- not that I expected any mind you, but the best way to stay  
current with them costs $$ that I'm not likely to fork over any time  

On May 3, 2006, at 3:36 PM, Michael Mauws wrote:

> Thanks yet again, Dieter, particularly for putting to rest a few of
> the "solutions" I found while poking around. As you might expect, I've
> been continuing to explore this problem and, at the moment, I'm
> zeroing in on rollback segments. If this is indeed the problem, it
> would fit well with what you're saying....
> ...michael

More information about the Zope-DB mailing list