[Zope-dev] Test runner: layers, subprocesses, and tear down

[Benji York]

I'm working on making the zope.testing test runner run tests in
parallelized subprocesses.  The option will likely be spelled -j N,
where N is the maximum number of processes.

I have it basically working, but have noticed a couple odd corners of
the test runner that I'd like to clean up.  They may be controversial,
so I'll ask about them here first.

I'd like to 1) remove the layer tear-down mechanism entirely, and 2)
make (almost) all layers run in a subprocess.

I want to do #1 because it would simplify the test runner code and no
one seems to be using the functionality anyway.  It also appears from
reading the code that any tests run in a subprocess (and most are) will
never exercise the tear-down mechanism anyway.

#2 will add some process start-up overhead, but it'll only be one more
process than is already started (and any reasonably-sized test corpus
already starts several processes for each test run).  The one exception
is for running with -D or with a pdb.set_trace() embedded in the code
under test.  For that case we need a switch to say "don't start any
subprocesses at all", I suspect that will be spelled -j0.

For motivation, some speed comparisons: running a particular test suite
with 3876 tests (mostly doctests, and mostly functional) without the
patch takes 6 minutes, 42 seconds; my branch runs the same tests in 3
minutes and 22 seconds (give or take) on a dual-core box with 3
simultaneous subprocesses.
-- 
Benji York
Senior Software Engineer
Zope Corporation