[Zope-dev] zope.security test failure on python2.7

[Gediminas Paulauskas]

2010/10/27 Tres Seaver <tseaver at palladion.com>:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 10/27/2010 07:58 AM, Zope Tests Summarizer wrote:
...
>> Subject: FAILED : Zope Buildbot / zopetoolkit-py2.7 slave-ubuntu64
>> From: jdriessen at thehealthagency.com
>> Date: Wed Oct 27 04:49:22 EDT 2010
>> URL: http://mail.zope.org/pipermail/zope-tests/2010-October/022426.html
>>
>> Subject: FAILED : Zope Buildbot / zopetoolkit-py2.7 slave-ubuntu32
>> From: jdriessen at thehealthagency.com
>> Date: Wed Oct 27 04:49:43 EDT 2010
>> URL: http://mail.zope.org/pipermail/zope-tests/2010-October/022427.html
>
> Our failures now seem to be all on Python 2.7 (progress, of a sort).
> Many of them fail due to the following attempt to use a proxy as the
> target of a weakref:
>
> - ----------------------------- %< -----------------------------
> File
> "/home/zope/.buildout/eggs/zope.security-3.7.4-py2.7-linux-i686.egg/zope/security/tests/test_standard_checkers.py",
> line 509, in zope.security.tests.test_standard_checkers.test_ABCMeta
> Failed example:
>    issubclass(PBar, Foo)
> Exception raised:
>    Traceback (most recent call last):
>      File "/usr/lib/python2.7/doctest.py", line 1253, in __run
>        compileflags, 1) in test.globs
>      File "<doctest
> zope.security.tests.test_standard_checkers.test_ABCMeta[6]>", line 1, in
> <module>
>        issubclass(PBar, Foo)
>      File "/usr/lib/python2.7/abc.py", line 151, in __subclasscheck__
>        if subclass in cls._abc_cache:
>      File "/usr/lib/python2.7/_weakrefset.py", line 69, in __contains__
>        return ref(item) in self.data
>    TypeError: cannot create weak reference to
> 'zope.security._proxy._Proxy' object
>
>  Ran 124 tests with 1 failures and 0 errors in 0.433 seconds.
> Tearing down left over layers:
>  Tear down zope.testrunner.layer.UnitTests in 0.000 seconds.
> - ----------------------------- %< -----------------------------
>
> Observations:
>
> - - Proxies can't be weakref targets under either Python 2.6 or 2.7.
>
> - - Under Python 2.7, the ABCMeta._abc_cache involved in the traceback
>  is being morphed from a normal set (per the __new__ of the metaclass)
>  to a _weakrefset.WeakSet.  Such sets cannot hold non-weakref-target
>  objects.

I investigated this issue today, and committed a fix
http://zope3.pov.lt/trac/changeset/121714

It removes a check issubclass(PBar, Foo), where PBar is a proxy of class Bar.

This worked in Python 2.6, but not in Python 2.7 anymore because ABC
implementation was changed to use a WeakSet instead of set for the
cache, and _Proxy objects cannot be ref'ed. However, other objects
cannot be ref'ed either, the first argument to issubclass has to be a
type (see http://bugs.python.org/issue10569).

http://docs.zope.org/zopetoolkit/releases/overview-1.1.html#python-versions
links to http://bugs.python.org/issue10360 as the underlying problem.
I have applied the patch, but the test fails in the same place with a
different stacktrace:

File "/home/menesis/.buildout/eggs/zope.security-3.8.1-py2.7-linux-i686.egg/zope/security/tests/test_standard_checkers.py",
line 509, in zope.security.tests.test_standard_checkers.test_ABCMeta
Failed example:
    issubclass(PBar, Foo)
Exception raised:
    Traceback (most recent call last):
      File "/usr/lib/python2.7/doctest.py", line 1254, in __run
        compileflags, 1) in test.globs
      File "<doctest
zope.security.tests.test_standard_checkers.test_ABCMeta[6]>", line 1,
in <module>
        issubclass(PBar, Foo)
      File "/home/menesis/src/zope/zopetoolkit-1.1/python/lib/python2.7/abc.py",
line 171, in __subclasscheck__
        cls._abc_cache.add(subclass)
      File "/home/menesis/src/zope/zopetoolkit-1.1/python/lib/python2.7/_weakrefset.py",
line 84, in add
        self.data.add(ref(item, self._remove))
    TypeError: cannot create weak reference to
'zope.security._proxy._Proxy' object

So the patch, that we were waiting for to be released with python
2.7.2, does not help.

So I removed the (incorrect) issubclass call, and the rest of the test
passes, showing that ABC checker works correctly. Even if it would be
good that issubclass worked, I think that mixing ABCs and Zope
interfaces is very unlikely anyway.

But having one test that is known to fail, and consequently not test
all of ZTK at all is wrong. Now ZTK builders on python 2.7 can be
added/enabled again. Please do so.

Gediminas