[Zope-dev] New-style ExtensionClass, ZODB 3.3, and Zope 2.8 status

[Jim Fulton]

I spent what turned into a couple of weeks on new-style
ExtensionClass (NSEC).  This is a reimplementation of ExtendionClass
as a new-style type.  Actually, ExtensionClass.ExtensionClass is a
subclass of type and ExtensionClass.Base is a subclass of object
whose type is ExtensionClass.ExtensionClass. :)

NSEC combines the features of ExtensionClass:

- __of__

- __class_init__

- inheritedAttribute

- traditional method-lookup order (for better or worse)

with the features of new-style classes, including:

- descriptors,

- slots,

- garbage collection

NSEC is mostly backward compatible with old ExtensionClass.  The
major backward incompatability is that method attributes are handled
differently. Unlike old EC, NSEC doesn't provide it's own method
type. It uses standard Python methods.  Old EC methods got their
attributes from theor instances by combining their names with the
attribute names. For example, if you ask for the '__roles__'
attribute on a method named 'foo', the method will try to get the
value my looking up the 'foo__roles__' attribute on the bound
instance.  Standard Python methods don't have this behavior. They
get their attributes from their underlying functions.  This has
major implications for Zope 2's security machinery. More on this
later.

A few months ago, Jeremy Hylton and Tim Peters created a version of
ZODB 3 that didn't use ExtensionClass.  This was refered to as "ZODB
3.3" and was implemented on the zodb33-devel-branch (pronounced
"devil branch" ;).  I've extended this branch to all of Zope and
integrated NSEC into the branch.  Despite it's name, this branch
should be thought of as the Zope 2.8 development branch.

Work is underway to add multi-version concurrency control (no read
conflicts :) to ZODB 3. This work is underway and will land in ZODB
3.3.

During the time working on this, a plan for Zope 2.8 emerged that I
think is pretty reasonable.  Zope 2.8 will include:

- NSEC

- ZODB 3.3

and whetever other changes happen to be made while the work on NSEC
and ZODB 3.3 is progressing.

Currently, on the zodb33-devel-branch, Zope 2 runs and the great
majority of it's tests pass with NSEC and ZODB 3.3. It you'd like to
see this in action, feel free to check Zope out on the branch and
play with it.

I'm anxious to get this work merged into the Zope 2 CVS head, but
have run into a problem that has stopped me for now. After 2 weeks, I
really needed to get back to work on Zope 3.

Gory details

   The issue preventing me from finishing this work is that a
   significant (though not massive) refactoring of the Zope 2 security
   machinery is needed to accomidate NSEC.  This happens to be a
   refactoring that is loooong overdue. (It should have been done
   before Principia was created.)

   Long ago (even before Principia), our security model consisted of
   users, roles and objects. Objects required roles. Users had roles.
   To provide greater control, we added another layer of indirection:
   permissions. Now, object's require permissions to be accessed.
   Roles are granted permissions. Users are assigned roles.

   When we introduced permissions, we didn't want to refactor existing
   code, so we created permission-roles.  Permission roles are complex
   computed attributes. When you access a __roles__ attribute on an
   object, a complex computation, making heavy-use of acquisition, is
   performed by to find all of the roles that have the required
   permission for the object. What can I say, this is a travesty. :)

   When we ask for the __roles__ of a method of an old-style
   ExtensionClass instance, the method turns around and accesses an
   attribute on the instance, triggering the roles computation.

   NSEC methods are just standard Python methods.  We can store
   security assertions on them, but we can't use computed attributes.

   The whole computed roles hack won't work.  I could make it work my
   making the NSEC implementation much more complex, but this would
   be wrong. The hack should have been done away with ages ago.

   We need to refactor the way security assertions (permission
   settings) are stored and accessed.  We need to store required
   permissions (__permissions__) on objects. When we need to figure
   out roles, we need to compute them at a higher level. (For
   example, we could compute the needed roles in the zope security
   policy It's likely that we can speed security checks when doing
   this.)

I estimate that the necessary refactoring would take me 3-5
days. The vast majority of the required time will be spent writing
tests.  I really need to focus on Zope 3 for a while, so I may not
be able to get back to this soon.  I think that this is an area
where some volunteers could make a big difference.  I'd be happy to
work with some folks on this.

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org