[Zope] Zope 2.8 and conflict errors

Chris McDonough chrism at plope.com
Mon Aug 1 10:55:04 EDT 2005


Hi Maciej,

I can't speak to the 500 errors you're receiving but Zope 2.8.0 ships
with a "TemporaryStorage" implementation (used by sessions in the
default configuration) which does not implement MVCC.  The one in 2.8.1
(now in beta) and better will implement MVCC, so you *should* see
performance improve (or at least see fewer conflict errors while using
sessioning).  I'm keenly interested in your results, so please let me
know.

- C


On Mon, 2005-08-01 at 12:21 +0200, Maciej Wisniowski wrote:
> Hi!
> 
> Does anybody tried 2.8 and can say something about conflict errors?
> 
> I've done few tests and results are not good.
> 
> We've checked one of our applications on Zope 2.7.2 and the same
> application on Zope 2.8. We have simulated 50-65 users executing the
> same path in application. There was total count 1500 pages visited
> during test.
> 
> With Zope 2.7.2 we've received about 39 internal server errors
> HTTP/1.1 500.
> 
> Errors in Zope 2.7.2 are mainly ReadConflictErrors with
> DCOracle2.DA.Procedure like:
> 
> ------
> 2005-08-01T09:54:08 INFO(0) TM:65540 Failed to abort object
> Traceback (most recent call last):
>   File "/opt/Zope/2.7.2/lib/python/ZODB/Transaction.py", line 393, in 
> _commit_error
>     j.abort(o, self)
>   File "/opt/Zope/2.7.2/lib/python/ZODB/Connection.py", line 561, in 
> setstate
>     invalid = self._is_invalidated(obj)
>   File "/opt/Zope/2.7.2/lib/python/ZODB/Connection.py", line 590, in 
> _is_invalidated
>     raise ReadConflictError(object=obj)
> ReadConflictError: database read conflict error (oid 00000000000299de, 
> class Products.DCOracle2.DA.Procedure)
> ------
> 
> Other errors are like "Shouldn't load state xxx when
> connection is closed" but these are known bugs in older ZODB version.
> 
> 
> Zope 2.8 supports MVCC so we hoped that the number of
> errors decrease but... with Zope 2.8 we get about 129 internal
> server errors HTTP/1.1 500 and additionally a lot of 402 errors 
> (unauthorized).
> 
> Errors 402 are because of internal server errors on login page
> (testing application continues test without checking if user has been
> properly logged in).
> 
> In the event.log there are no conflict errors saved :-/ ,
> only unauthorized. We get the error from client responses
> and from console.
> 
> I don't know what happens. May it be because of using old
> data.fs file (from 2.7.2) with Zope 2.8?
> 
> Anybody has similiar experiences? Is Zope 2.8 useable??
> 
> 
> Below few errors from the console (runzope) and from user browser:
> 
> ----------------------------------------------------------------
> 2005-08-01 10:46:22 INFO ZODB conflict error at /crProd/acl_users/login 
> (16 conflicts since startup at 2005-08-01T10:45:58)
> 2005-08-01 10:46:22 BLATHER Conflict traceback
> Traceback (most recent call last):
>   File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 113, in 
> publish
>     request, bind=1)
>   File "/opt/Zope/2.8.0/lib/python/ZPublisher/mapply.py", line 88, in mapply
>     if debug is not None: return debug(object,args,context)
>   File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 40, in 
> call_object
>     result=apply(object,args) # Type s<cr> to step into published object.
>   File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 
> 311, in __call__
>     return self._bindAndExec(args, kw, None)
>   File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 
> 348, in _bindAndExec
>     return self._exec(bound_data, args, kw)
>   File 
> "/opt/Zope/2.8.0/lib/python/Products/PythonScripts/PythonScript.py", 
> line 323, in _exec
>     result = f(*args, **kw)
>   File "Script (Python)", line 3, in login
>   File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 
> 1214, in __getattr__
>     v = self.get(key, default, returnTaints=returnTaints)
>   File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 
> 1174, in get
>     if callable(v): v = v()
>   File 
> "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", 
> line 93, in getSessionData
>     return self._getSessionDataObject(key)
>   File 
> "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", 
> line 180, in _getSessionDataObject
>     ob = container.new_or_existing(key)
>   File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", 
> line 844, in new_or_existing
>     item = self.get(key, _marker)
>   File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", 
> line 491, in get
>     item = self._move_item(k, current_ts, default)
>   File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", 
> line 297, in _move_item
>     abucket = self._data.get(ts, None) # XXX ReadConflictError hotspot
>   File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 704, in 
> setstate
>     self._setstate(obj)
>   File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 757, in 
> _setstate
>     self._load_before_or_conflict(obj)
>   File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 768, in 
> _load_before_or_conflict
>     raise ReadConflictError(object=obj)
> ReadConflictError: database read conflict error (oid 0x0a, class 
> BTrees._IOBTree.IOBTree)
> ----------------------------------------------------------------
> 
> error in /crProd/acl_users/login - is with request.SESSION statement.
> 
> ----------------------------------------------------------------
> 2005-08-01 10:46:21 BLATHER Conflict traceback
> Traceback (most recent call last):
>   File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 113, in 
> publish
>     request, bind=1)
>   File "/opt/Zope/2.8.0/lib/python/ZPublisher/mapply.py", line 88, in mapply
>     if debug is not None: return debug(object,args,context)
>   File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 40, in 
> call_object
>     result=apply(object,args) # Type s<cr> to step into published object.
>   File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 
> 311, in __call__
>     return self._bindAndExec(args, kw, None)
>   File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 
> 348, in _bindAndExec
>     return self._exec(bound_data, args, kw)
>   File 
> "/opt/Zope/2.8.0/lib/python/Products/PythonScripts/PythonScript.py", 
> line 323, in _exec
>     result = f(*args, **kw)
>   File "Script (Python)", line 3, in login
>   File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 
> 1214, in __getattr__
>     v = self.get(key, default, returnTaints=returnTaints)
>   File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 
> 1174, in get
>     if callable(v): v = v()
>   File 
> "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", 
> line 93, in getSessionData
>     return self._getSessionDataObject(key)
>   File 
> "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", 
> line 180, in _getSessionDataObject
>     ob = container.new_or_existing(key)
>   File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", 
> line 844, in new_or_existing
>     item = self.get(key, _marker)
>   File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", 
> line 491, in get
>     item = self._move_item(k, current_ts, default)
>   File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", 
> line 280, in _move_item
>     self._housekeep(current_ts)
>   File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", 
> line 939, in _housekeep
>     self._finalize(now)
>   File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", 
> line 547, in _finalize
>     last_finalized = self._last_finalized_timeslice()
>   File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", 
> line 1076, in __call__
>     return self.value
>   File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 704, in 
> setstate
>     self._setstate(obj)
>   File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 757, in 
> _setstate
>     self._load_before_or_conflict(obj)
>   File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 768, in 
> _load_before_or_conflict
>     raise ReadConflictError(object=obj)
> ReadConflictError: database read conflict error (oid 0x09, class 
> Products.Transience.Transience.Increaser)
> -----------------------------------------------------------
> 
> or
> 
> -----------------------------------------------------------
> 2005-08-01 10:46:21 BLATHER Conflict traceback
> Traceback (most recent call last):
>   File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 113, in 
> publish
>     request, bind=1)
>   File "/opt/Zope/2.8.0/lib/python/ZPublisher/mapply.py", line 88, in mapply
>     if debug is not None: return debug(object,args,context)
>   File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 40, in 
> call_object
>     result=apply(object,args) # Type s<cr> to step into published object.
>   File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 
> 311, in __call__
>     return self._bindAndExec(args, kw, None)
>   File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 
> 348, in _bindAndExec
>     return self._exec(bound_data, args, kw)
>   File 
> "/opt/Zope/2.8.0/lib/python/Products/PythonScripts/PythonScript.py", 
> line 323, in _exec
>     result = f(*args, **kw)
>   File "Script (Python)", line 3, in login
>   File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 
> 1214, in __getattr__
>     v = self.get(key, default, returnTaints=returnTaints)
>   File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 
> 1174, in get
>     if callable(v): v = v()
>   File 
> "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", 
> line 93, in getSessionData
>     return self._getSessionDataObject(key)
>   File 
> "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", 
> line 180, in _getSessionDataObject
>     ob = container.new_or_existing(key)
>   File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", 
> line 847, in new_or_existing
>     self[key] = item
>   File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", 
> line 435, in __setitem__
>     length = self._length() # XXX ReadConflictError hotspot
>   File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", 
> line 1119, in __call__
>     return self.value
>   File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 704, in 
> setstate
>     self._setstate(obj)
>   File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 757, in 
> _setstate
>     self._load_before_or_conflict(obj)
>   File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 768, in 
> _load_before_or_conflict
>     raise ReadConflictError(object=obj)
> ReadConflictError: database read conflict error (oid 0x08, class 
> Products.Transience.Transience.Length2)
> -----------------------------------------------------------
> 
> or
> 
> -----------------------------------------------------------
> 2005-08-01 10:46:21 BLATHER Conflict traceback
> Traceback (most recent call last):
>   File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 113, in 
> publish
>     request, bind=1)
>   File "/opt/Zope/2.8.0/lib/python/ZPublisher/mapply.py", line 88, in mapply
>     if debug is not None: return debug(object,args,context)
>   File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 40, in 
> call_object
>     result=apply(object,args) # Type s<cr> to step into published object.
>   File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 
> 311, in __call__
>     return self._bindAndExec(args, kw, None)
>   File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 
> 348, in _bindAndExec
>     return self._exec(bound_data, args, kw)
>   File 
> "/opt/Zope/2.8.0/lib/python/Products/PythonScripts/PythonScript.py", 
> line 323, in _exec
>     result = f(*args, **kw)
>   File "Script (Python)", line 3, in login
>   File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 
> 1214, in __getattr__
>     v = self.get(key, default, returnTaints=returnTaints)
>   File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 
> 1174, in get
>     if callable(v): v = v()
>   File 
> "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", 
> line 93, in getSessionData
>     return self._getSessionDataObject(key)
>   File 
> "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", 
> line 180, in _getSessionDataObject
>     ob = container.new_or_existing(key)
>   File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", 
> line 844, in new_or_existing
>     item = self.get(key, _marker)
>   File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", 
> line 491, in get
>     item = self._move_item(k, current_ts, default)
>   File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", 
> line 297, in _move_item
>     abucket = self._data.get(ts, None) # XXX ReadConflictError hotspot
>   File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 704, in 
> setstate
>     self._setstate(obj)
>   File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 757, in 
> _setstate
>     self._load_before_or_conflict(obj)
>   File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 768, in 
> _load_before_or_conflict
>     raise ReadConflictError(object=obj)
> ReadConflictError: database read conflict error (oid 0x0a, class 
> BTrees._IOBTree.IOBTree)
> -----------------------------------------------------------
> 
> or
> 
> -----------------------------------------------------------
> 2005-08-01 10:47:55 INFO ZODB conflict error at 
> /crProd/cr/centralne/procName (87 conflicts since startup at 
> 2005-08-01T10:45:58)
> 2005-08-01 10:47:55 BLATHER Conflict traceback
> Traceback (most recent call last):
>   File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 119, in 
> publish
>     transactions_manager.commit()
>   File "/opt/Zope/2.8.0/lib/python/Zope2/App/startup.py", line 215, in 
> commit
>     transaction.commit()
>   File "/opt/Zope/2.8.0/lib/python/transaction/_manager.py", line 84, in 
> commit
>     self.get().commit(sub)
>   File "/opt/Zope/2.8.0/lib/python/transaction/_transaction.py", line 
> 382, in commit
>     self._saveCommitishError() # This raises!
>   File "/opt/Zope/2.8.0/lib/python/transaction/_transaction.py", line 
> 380, in commit
>     self._commitResources()
>   File "/opt/Zope/2.8.0/lib/python/transaction/_transaction.py", line 
> 428, in _commitResources
>     rm.tpc_vote(self)
>   File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 637, in 
> tpc_vote
>     s = vote(transaction)
>   File "/opt/Zope/2.8.0/lib/python/ZEO/ClientStorage.py", line 893, in 
> tpc_vote
>     return self._check_serials()
>   File "/opt/Zope/2.8.0/lib/python/ZEO/ClientStorage.py", line 877, in 
> _check_serials
>     raise s
> ConflictError: database conflict error (oid 0x0299de, class 
> Products.DCOracle2.DA.Procedure, serial this txn started with 
> 0x035f0726c7bfca88 2005-08-01 08:38:46.816215, serial currently 
> committed 0x035f072fed78ebee 2005-08-01 08:47:55.657582)
> -----------------------------------------------------------
> 
> User receives errors like:
> -----------------------------------------------------------------
> Response:
> HTTP/1.1 500 Internal Server Error
> Server: Zope/(Zope 2.8.0-final, python 2.3.5, linux2) ZServer/1.1
> Date: Fri, 29 Jul 2005 12:53:00 GMT
> Bobo-Exception-Line: 877
> Content-Length: 2020
> Bobo-Exception-Value: See the server error log for details
> Content-Language: pl
> Bobo-Exception-File: ClientStorage.py
> Bobo-Exception-Type: ZODB.POSException.ConflictError
> 
> (...)
> 
> <TD WIDTH="90%">
>   <H2>Site Error</H2>
>   <P>An error was encountered while publishing this resource.
>   </P>
>   <P><STRONG>ZODB.POSException.ConflictError</STRONG></P>
>   Sorry, a site error occurred.<p><p>Traceback (innermost last):
> <ul>
> 
> Module ZPublisher.Publish, line 187, in publish_module_standard
> <li>  Module ZPublisher.Publish, line 161, in publish</li>
> <li>  Module ZPublisher.Publish, line 161, in publish</li>
> <li>  Module ZPublisher.Publish, line 161, in publish</li>
> <li>  Module ZPublisher.Publish, line 151, in publish</li>
> <li>  Module Zope2.App.startup, line 158, in zpublisher_exception_hook</li>
> <li>  Module ZPublisher.Publish, line 119, in publish</li>
> <li>  Module Zope2.App.startup, line 215, in commit</li>
> <li>  Module transaction._manager, line 84, in commit</li>
> <li>  Module transaction._transaction, line 382, in commit</li>
> <li>  Module transaction._transaction, line 380, in commit</li>
> <li>  Module transaction._transaction, line 428, in _commitResources</li>
> <li>  Module ZODB.Connection, line 637, in tpc_vote</li>
> <li>  Module ZEO.ClientStorage, line 893, in tpc_vote</li>
> <li>  Module ZEO.ClientStorage, line 877, in _check_serials</li>
> </ul>ConflictError: database conflict error (oid 0x0299de, class 
> Products.DCOracle2.DA.Procedure, serial this txn started with 
> 0x035ef744f3d9b366 2005-07-29 12:52:57.152436, serial currently 
> committed 0x035ef744fe4886cc 2005-07-29 12:52:59.597650)
> </p>
>   <HR NOSHADE>
>   <P>Troubleshooting Suggestions</P>
> (...)
> -----------------------------------------------------------------
> 
> 



More information about the Zope mailing list