[Zope] Large BTreeFolder2 batching/pagination
feedreader at yahoo.com
Sun Nov 8 07:17:59 EST 2009
I take your suggestion to find a Catalog-based solution. I have set the subtransactions very low (100).
But I have got the following error:
Traceback (innermost last):
Module ZPublisher.Publish, line 119, in publish
Module ZPublisher.mapply, line 88, in mapply
Module ZPublisher.Publish, line 42, in call_object
Module Products.ZCatalog.ZCatalog, line 348, in manage_catalogFoundItems
Module Products.ZCatalog.ZCatalog, line 800, in ZopeFindAndApply
Module Products.ZCatalog.ZCatalog, line 559, in catalog_object
Module transaction._manager, line 110, in savepoint
Module transaction._transaction, line 312, in savepoint
Module transaction._transaction, line 309, in savepoint
Module transaction._transaction, line 737, in __init__
Module ZODB.Connection, line 1034, in savepoint
Module ZODB.Connection, line 543, in _commit
Module ZODB.Connection, line 571, in _store_objects
Module ZODB.Connection, line 1168, in store
IOError: [Errno 28] No space left on device
I am guessing that this process is using the /tmp directory which unfortunately was incorrectly sized at only 1.9Gb. Can someone confirm this?
Then either I must find a way to change the /tmp size, or is there a solution through Zope?
--- On Mon, 11/2/09, Andreas Jung <lists at zopyx.com> wrote:
> We have not been successful to create a Catalog of this many objects.
> The process seemed to time out after many hours, probably
> hardware-bound. Also, the size of the resulting ZODB is of concern,
> but we may try again with the Catalog on a mounted database.
we have ZCatalogs with millions of objects..likely you are indexing
everything within one big transaction
without using savepoints or subtransactions..
More information about the Zope