[Zope-DB] Zope, DCOracle2, concurrent transactions, etc.
Matthew T. Kromer
matt at bane.mi.org
Thu May 4 07:44:26 EDT 2006
I'm pretty confident that the db component (the C layer) doesn't have
any GIL problems. However, the DA component (the Zope layer) *may*
have all sorts of unusual failure modes.
Debugging transaction concurrency problems is always tricky, but keep
in mind there are Python layer abstractions that get and release
locks as well.
I'm speculating based on the history of your discussion thread that
you're running out of Zope db connections. The way it works in Zope
is somewhat unusual; there are usually NOT enough object database
connections if you run more than four threads (unless someone fixed
this way back when, the default was 7). Threads in Zope wait in line
for a zodb connection to become available. Under normal
circumstances they don't have to wait, but in yours I think they are
waiting. Are you trying to run with more than four threads? If so
you might want to tweak the pool_size parameter up in the DB class of
lib/python/ZODB/DB.py. There are negative consequences of doing this
(ie each DB connection cache gobbles memory) but on the upside you
might be able to avoid stalling out Zope.
This won't provide you relief the way you might hope; if the problem
is that threads are entering a long running transaction and waiting
on Oracle to complete, all you're doing is adding a few more feet to
the rope before Zope hangs itself up again.
DCOracle2 does support event tracing, so you could turn on the trace
log to see what the module is doing when Zope is becoming
nonresponsive. It probably will say that it's waiting in a fetch or
execute statement, so that won't be all too terribly helpful but it
MIGHT indicate that there's a problem with handles (some versions of
DCOracle2 leaked a handle -- make sure you have the latest code from
the CVS repository).
Also, the C layer isn't capable of coping with some result types very
well (if at all) ie using Oracle in object mode or returning table
results from stored procedures, etc.
Also, the PYTHON layer code that currently exists as the DA *does*
leak connections if Oracle closes the connection. This is something
Chris Withers tried to fix, but I think he was never happy with the
solution and so it remains on a branch.
I haven't done any DCO2 development in a little over two years; I
hate to abandon the code but I don't run any current Oracles at home
and its a fairly intensive time commitment to just set up a
development environment. Oracle never gave me any freebies either, /
sniff -- not that I expected any mind you, but the best way to stay
current with them costs $$ that I'm not likely to fork over any time
On May 3, 2006, at 3:36 PM, Michael Mauws wrote:
> Thanks yet again, Dieter, particularly for putting to rest a few of
> the "solutions" I found while poking around. As you might expect, I've
> been continuing to explore this problem and, at the moment, I'm
> zeroing in on rollback segments. If this is indeed the problem, it
> would fit well with what you're saying....
More information about the Zope-DB