[Zope-dev] BTreeFolder2.objectIds() - accessing _tree.keys() slow

[Shane Hathaway]

sean.upton at uniontrib.com wrote:
> I have very large BTreeFolder2 (CMFMember via BaseBTreeFolder in Archetypes)
> - has about 260k items in _tree - objectIds() is painfully slow, as is
> self._tree.keys() - I've casually observed using the meta type index to get
> the object ids is many orders of magnitude faster.
> 
> Hacking objectIds() as follows (diff against trunk pasted inline) - gettting
> ids off of the meta type index for all used meta types - seems to make
> things much quicker.  Two questions:

Are you sure this actually works?  _mt_index.keys() is supposed to 
provide a list of all meta_types used in the folder.  To get the object 
ids from it, you'd need something like this:

ids = []
for d in self._mt_index.values():
   ids.extend(d.keys())

The structure of _mt_index is documented in a comment:

     _mt_index = None  # OOBTree: { meta_type -> OIBTree: { id -> 1 } }

(It's strange that I chose to use an OIBTree instead of an OOTreeSet. 
Maybe I didn't know about the set support in the BTree module at the time.)

Shane