[Grok-dev] Re: Performance of OrderedContainer

[Martijn Faassen]

Sebastian Ware wrote:
> I was looking at the trunk of grok because I need to use 
> OrderedContainer. But, if I add say 1 million objects to an 
> OrderedContainer, won't it become very slow due to the implementation 
> with PersistentList? Say that I want to show the last 100 of the 
> objects, I would have to return all 1 million keys.

Hm, thanks for that review, I think you are correct. Once the million 
keys are all in memory it'll be okay, but loading all those million keys 
into memory from the ZODB isn't very efficient.

I think it should use a BTree of some kind for the order information 
(_order) to prevent this problem. BTrees are however automatically 
sorted, so the design isn't obvious. The original OrderedContainer in 
Zope 3 seems to also use PersistentList, so that code is no help here.

Ideas, patches welcome. :)

Regards,

Martijn