[Zope] ZCatalog Strategy

[Jonathan]

One factor is the amount of ram that you have.  If you have enough ram to 
fit the entire catalog (with indexes)  then your search time is very fast. 
I would err on the side of less meta-data.  We currently run about 1 million 
entries in a zcatalog with very little meta-data and then use 
restrictedTraverse with the object ids returned by the zcatalog search to 
get the fields we need.  We typically have sub 1 second searches using this 
approach (but we have very fast scsi disks in striped raid configuration).

With only 10k records in the catalog you could try both configurations (very 
easy to clear and then reload the catalog) and time them to see which gives 
you better performance.

hth & good luck!

Jonathan


----- Original Message ----- 
From: "Mark Gibson" <mark at dimensional.com>
To: <zope at zope.org>
Sent: Wednesday, March 22, 2006 9:28 PM
Subject: [Zope] ZCatalog Strategy


> I'm struggling to weigh the cost of getObject() vs. the cost of adding 
> more metadata to the catalog.  I'll explain my situation.
>
> I have 10,000 widgets cataloged.  I do a path and date query that returns 
> me maybe 12 of these.  Then I have a choice of calling 
> getObject().getData() on each of these, or I could add getData to the 
> catalog metadata.
>
> Does the cost of calling getObject() for a dozen objects justify creating 
> a new metadata field?
>
> More generally how does a large amount of metadata in the catalog affect 
> performance of queries?
>
> The wisdom of those more knowledgeable than me would be appreciated.
>
> Mark
> _______________________________________________
> Zope maillist  -  Zope at zope.org
> http://mail.zope.org/mailman/listinfo/zope
> **   No cross posts or HTML encoding!  **
> (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce
> http://mail.zope.org/mailman/listinfo/zope-dev )
>