[Zope-dev] Memory leak, the resolution

[Shane Hathaway]

Leonardo Rochael Almeida wrote:
>>Why?  Although the catalog manages those DateTime objects, the DateTimes 
>>are in many ZODB records.
> 
> The problem is: there should be a more direct relation between the
> number of allowed objects in the cache and the number of refcounts. As
> you say, the DateTimes are in ZODB Records, so if I see a DateTime
> refcount of, say 120k, I should be seeing at least that many objects in
> the caches, but I was seeing under 3k objects, unless there's something
> else that could be counted as a refcount multiplying factor, say if
> something was keeping a reference to the class for every instance
> besides the instance itself.

Well, DateTimes do not derive from ZODB.Persistent, so they may be part 
of a ZODB object, but are never an entire ZODB object.  If you have 3k 
objects in the cache and there are 120k DateTimes floating around, you 
must have an average of 40 DateTimes per loaded object.  That's huge... 
but if you have FieldIndexes that index DateTimes, it wouldn't surprise 
me.  We at Zope Corp. didn't even realize what a problem this could be 
until recently.

> So we set the cache limit to 1000 objects, and to our amazement, Zope
> sat comfortably at 150MB RSS, and the performance was very good. Some
> heavy ZCatalog pages still send Zope for a walk for a few seconds (for
> those and all other requests) but the site becomes responsive again in
> under 20 seconds. Not only that, but the DateTime refcount isn't even
> among the top 5 (the BTrees are winning there now).
> 
> So we now have breathing space to do the great ZCatalog reform.

Excellent.  The average object size sounds very large, though: start 
with 150 MB, give about 15 MB for non-ZODB code and data, divide by 4 (4 
threads, each with independent copies of the same data), and divide by 
1000 objects.  That comes to 33-35k per object.  That's much larger than 
most Zopes I've seen.

BTW, if you're using ZEO, raise ZEO cache size at the same time you 
lower the ZODB cache size.  The ZEO cache can compensate for the 
performance loss.  The default ZEO cache size is 20 MB, but I prefer 200 
MB with today's Zope apps.

Shane