[Zope-dev] Infuriating ZClass registry Heisenbug

[Evan Simpson]

This past weekend I migrated a bunch of ZClasses and the web site that 
uses them from a scratch Zope instance into my production instance. 
Both run off the same 2.6 CVS branch checkout, connecting to ZEO servers 
running from the same ZEO trunk checkout.

For a while after starting the server, everything works fine.  After an 
unpredictable time period ranging from minutes to hours, suddenly my 
ZClass instances become broken, and I start seeing this in my log, over 
and over:

2002-12-10T11:33:18 ERROR(200) ZODB Couldn't load state for 
'\x00\x00\x00\x00\x00\x01\x1f\xc9'
Traceback (innermost last):
   Module ZODB.Connection, line 533, in setstate
ImportError: No module named */rJ6Q2CNqMJDSN4Okpwiuw==

The object that fails to load state is a BTree Bucket belonging to the 
ZClass registry, and "/rJ6Q2CNqMJDSN4Okpwiuw==" is the GUID of one of my 
ZClasses.  If I delete that ZClass, it complains about a different one.

If I restart the server, everything works again, for a while, but I get 
the following in my log:

2002-12-10T11:36:49 INFO(0) ZCS:2443 Connected to storage
------
2002-12-10T11:36:49 INFO(0) ZODB Opening database for mounting: 
'144865536_1012175858.273945'
------
2002-12-10T11:36:49 INFO(0) ZODB Mounted database 
'144865536_1012175858.273945' at /temp_folder
------
2002-12-10T11:36:50 ERROR(200) ZODB Couldn't load state for 
'\x00\x00\x00\x00\x00\x01\x1f\xc9'
Traceback (innermost last):
   Module ZODB.Connection, line 533, in setstate
ImportError: No module named */rJ6Q2CNqMJDSN4Okpwiuw==
------
2002-12-10T11:36:50 ERROR(200) Zope A problem was found when checking 
the global product registry.  This is probably due to a Product being 
uninstalled or renamed.  The traceback follows.
Traceback (innermost last):
   Module OFS.Application, line 218, in checkGlobalRegistry
   Module ZODB.Connection, line 533, in setstate
ImportError: No module named */rJ6Q2CNqMJDSN4Okpwiuw==
------
2002-12-10T11:36:50 INFO(0) Zope Beginning attempt to rebuild the global 
ZClass registry.
------
2002-12-10T11:36:52 INFO(0) Zope The global ZClass registry has 
successfully been rebuilt.
------
2002-12-10T11:36:52 INFO(0) ZServer HTTP server started at Tue Dec 10 
11:36:52 2002
         Hostname: server1.shackworks.com
         Port: 8280

Help!