[Zope-dev] Circular dependency hell.

[Lennart Regebro]

<This is to a large part an angry rant. Sorry for that.>

In my effort to port zope.component to Python 3 I constantly encounter
one problem that is very hard to solve: Circular dependencies. It's
hard to port module A, when running the tests require module B to be
ported, and module B depends on module A. Or, as with setuptools, it
actually uses itself, so it has to work before you can test if it
works. (Sorry, that's just insane).

So far the main circularity was that everything depended on
zope.testing as a testrunner, zc.buildout for making the development
environment, and zope.testing obviously depended on zope.interface
etc. I solved that by also adding support for setuptools/distributes
testrunner and using that instead. That fixed zope.interface,
zope.event and zope.exception. These modules still have buildout
configurations if you want them, but you don't need buildout anymore.
zope.interface, zope.event and zope.exception can now be developed and
tested with only setuptools, you can run the tests with "python
setup.py test" both under Python 2 and 3.

But then the time came to zope.testing itself. It, rather annoyingly,
depends on itself. I did the same there, but almost all tests are even
more annoyingly, doctests, and depends on a custom version of doctest
included, etc, etc, etc. The result was hours and hours of minor
fiddling to find exactly what character in a doctest that made it
fail. So I've spent an inordinate amount of time on zope.testing, and
I have come to a conclusion about it:

== Suggestion 1 ==

I think we should deprecate zope.testing. Completely. There has been
some discussion about deprecating it for something else, but I think
we should just deprecate it. Just say "Don't use zope.testing, it's
pure evil". We can recommend another testrunner, and it seems nose is
winning the wars in the Python world there, but I don't think we
necessarily need to do that. Most testrunners will find the tests by
themselves, and the tests should be runnable with any testrunner, so
which one you use is a minor issue.

Instead the big problem with getting rid of zope.testing is that many
tests use other specific features like the doctest renormalizers etc.
These could possibly be extracted into separate modules so you can use
them with other testrunners. But zope.testing is a mess that builds on
two other messes, namely unittest (yes, it's a bloody mess, the API
makes no sense) and doctest (which is a horror of a mess). unittest
and doctest are horrid enough for us to not to make it worse with
zope.testing in addition.

The path forward there would simply be to stop depending on
zope.testing for all ZTK modules. That's probably a bit of work, but I
think it's worth it.



So, with that in mind I today went on to zc.buildout, trying to port
it to Python 3 by ripping out any usage of zope.testing. Also, the
standard development mode for zc.buildout is that you run a dev.py
script, that uses zc.buildout to build a buildout for zc.buildout.
Obviously, this circularity is the same as I encountered in
setuptools, and it's evil. But with zc.buildout it's easy to get rid
off at least as I could use setuptools testrunner instead. But then I
realize the zc.buildout needs zc.recipe.egg. And zc.recipe.egg depends
on zc.buildout. Hohum, another circular depedency.

== Suggestion 2 ==

Move the egg recipe into zc.buildout itself, and make zc.recipe.egg a
dummy package that just points at zc.buildout.recipe.egg (or whatever
we decide to call it). That solves that problem.


-- 
Lennart Regebro: http://regebro.wordpress.com/
Python 3 Porting: http://python-incompatibility.googlecode.com/
+33 661 58 14 64