[Grok-dev] Re: Iterativly emptying a grok.container shows weird behavior

[Philipp von Weitershausen]

Brandon Craig Rhodes wrote:
>> Volker might like it more if the container had 100,000 items in it,
>> where iterating over the keys rather than getting them all back as a
>> list would be much faster.
> 
> Nonsense.  If Volker is unhappy getting a huge list from .keys(), he
> can call .iterkeys() instead.  Please review the Python interface for
> mapping types; it says explicitly that .keys() returns:
> 
>    "a copy of a's list of keys"
> 
> They key word there is "copy" - the caller has a right to expect that
> the sequence of keys he has been returned is a "copy", that is, an
> object that now stands independent of the dictionary-like object that
> generated it.
> 
> Making the decision to turn .keys() into .iterkeys() unilaterally
> destroys the whole point of dictionary-like objects supporting both
> methods, and, as Volker rightly complains, it breaks many common
> Python idioms for working with dictionaries.

When the BTree mapping type was conceived, Python didn't know iterators 
and there was no iterkeys() method in the mapping interface. BTrees 
store large mappings that possibly contain very large objects 
efficiently. They do that by storing each node of the search tree as a 
separate object. That way, when you're accessing an item in the mapping, 
it only has to deserialize the nodes along the search path, and it 
doesn't have to deserialize no other object except for the one you're 
getting from the mapping. Tests have shown that this is relevant once 
the number of items in your mapping exceeds about 100 (which isn't that 
much, really).

So, it might appear as nonsense, but it's a combination of efficiency 
and legacy:

* keys() returns an iterable and not a list because it would be very 
inefficient to traverse the whole BTree to clobber up a list if you're 
only going to batch over a fraction of them.

* It's not called iterkeys() because our BTree implementation in the 
ZODB has been around for about 10 years.


-- 
http://worldcookery.com -- Professional Zope documentation and training