[Zope-dev] Test fixture concepts (was: Zope test layers, pytest, and test isolation)

[Martin Aspeli]

Hi,

On 26 March 2011 14:18, Wolfgang Schnerring <ws at gocept.com> wrote:

> Because, while test layers are nice because they have the above
> properties, I'm not too happy with the current implementation, namely
> the use (or is it abuse? ;-) of __bases__ and __name__, which leads
> very naturally to not-so-helpful implementation patterns. As a
> concrete example, some of the test layers of ZTK packages, most
> notably the successors of zope.app.testing.functional in
> zope.component, zope.app.appsetup, zope.app.wsgi, are implemented
> right now in a way that does not have properties 2+3.
>
> The most straightforward way to improve that situation would be to use
> plone.testing, since that provides a layer implementation that is both
> sane and easy to use, and has all the nice properties. (Maybe even
> fold some of it into zope.testrunner itself.)

I'd be happy to move layer.py from plone.testing to zope.testrunner for sure.

> But then I realized, that's a major undertaking, so we better think
> about what the actual goals are before changing lots of stuff. And
> then I heard people expressing interest in other test runners.
> (py.test and nose seem to be the biggest players, stdlib's
> unittest/unittest2 seems to be woefully behind in terms of test
> selection and other features like coverage or debugging. I for one
> happen to like zope.testrunner, but if there's a good alternative, why
> not combine our efforts?)
> I also found that it is rather hard to explain how to use test layers
> in a way so you actually get the above properties, so I started
> wondering whether there was a cleaner way to get them.

Personally, I rely a lot on the layer mechanism and would not want to lose it.

> So I guess I have three questions:
>
> 1. (The starting point:) How can we improve the test infrastructure
> situation of the ZTK packages?

I'd say having more reusable test layers would help.

> 2. (The main thing:) Are test layers, both the concept and the
> implementation as epitomized by plone.testing, the way to go or is
> there a cleaner alternative?

When we looked at how to improve Plone's testing story, we couldn't
find anything better. One thing I would like is a better way to
support "python setup.py test".

> 3. (But only tangentially, I don't want to sidetrack the discussion at
> hand nor start a flamewar:) Is there a compelling alternative to
> zope.testrunner that would make it worthwile to think about either
> generalizing the fixture support or even migrating away from
> zope.testrunner?

It's a good question to ask, though I think this would be a really big
undertaking for maybe only marginal gain (if any).

Martin