[Zope-Coders] More testing fixups on HEAD

Tres Seaver tseaver@zope.com
Sat, 13 Oct 2001 16:15:21 -0400


Our framework for running unit tests needs some help.  In particular,
we have cases on the head right now for which varying the order in which
tests are executed changes the outcomes;  this is *grossly* unacceptable.

In particular, I added a tweak to testrunner to look for a '.testinfo'
file when entering a directory;  if found, testrunner uses it to determine
the tree descent from that directory, rather than relying on listdir.
When I used '.testinfo' in 'lib/python' to suppress AccessControl (whose
tests were all breaking in odd ways, anyway) all other tests ran fine.
Many had been breaking strangely before, when run as part of an "all tests"
run (the "-a" option), but would run fine when invoked directly by selecting
their parent directory ("-d").

Assuming that AccessControl's tests were polluting the global space
somehow, I then injected them at the tail end of the run, and, mirabile
dictu, they run!

I suspect that the use of 'execfile( "framework.py" )' could be to
blame here, as the typical practice does nothing to protect the global
namespace.

I believe that we should terminate the use of execfile with extreme
prejudice, preferring instead to import a base class for our test
cases from Testing.common, to gain access to the same utilities.
The affected modules would be:

  - /lib/python/AccessControl

  - /lib/python/BTrees

  - /lib/python/DocumentTemplate

  - /lib/python/OFS

  - /lib/python/Products/PluginIndexes (I've already done these tests)

  - /lib/python/Products/PythonScripts

  - /lib/python/Products/ZCatalog

  - /lib/python/SearchIndex (deprecated -- perhaps we should filter out?)

  - /lib/python/StructuredText

Interestingly, it was these same modules which were breaking when
the AccessControl tests were run first (with the exception of BTrees
and DocumentTemplate).

If no one objects, I will finish removing the 'execfile' stuff on
Monday.

Tres.
-- 
===============================================================
Tres Seaver                                tseaver@zope.com
Zope Corporation      "Zope Dealers"       http://www.zope.com