[Zope] ZCatalog performance issues - catalogging objects takes ages

[Dieter Maurer]

Wankyu Choi wrote at 2003-3-31 19:47 +0900:
 > - It takes ages when cataloging even a small number of articles. 18 seconds
 > for cataloging 50 or so article objects with so little to index? Is it
 > normal? Can't imagine recataloging 20,000 objects.  For example, if you move
 > a thread from one NeoBoard instance to another, you have to uncatalog the
 > thread including  all its replies in NeoBoad A and catalog them in NeoBoard
 > B: cataloging a single article object takes more than 1 second. Don't think
 > it's normal... Or is it?

The connection between objects and cataloguing is very generic and
flexible but unfortunately also costly.

  Each index and each metadata entry causes one method to be called
  per document. If the call is expensive, this can slow things down
  considerably.

  Use only metadata entries, you really need.

We have about 5 metadata entries and about 7 indexes and can
index about 10 documents per second (on a 1.4 GH AMD).

Batching indexing can help a lot to improve efficiency.
We use Shane's QueueCatalog for this purpose.


Dieter