[Zope-dev] Re: Test runner: layers, subprocesses, and tear down

Benji York benji at zope.com
Fri Jul 4 11:50:34 EDT 2008


On Thu, Jul 3, 2008 at 5:22 PM, Benji York <benji at zope.com> wrote:
> I'm working on making the zope.testing test runner run tests in
> parallelized subprocesses.  The option will likely be spelled -j N,
> where N is the maximum number of processes.

The branch (svn+ssh://svn.zope.org/repos/main/zope.testing/branches/benji-parallelize-subprocesses)
is feature complete.  I basically did a very simple transformation that
resulted in the runner spawning subprocesses in threads, several at a
time, instead of spawning them serially.  The patch is less than 250
lines.  Any critiques of the changes would be appreciated.

No changes to the tear-down mechanism were made.  All existing tests
pass without modification.  There aren't yet any tests for the new
functionality.  It may be tricky to test, so I have to think about that
bit.

I found that to eliminate nearly all CPU idle time, I had to use -j4 on
my two core laptop.

For a particular test corpus on a 4 core machine -j1 (the default) takes
about 7 minutes -j6 takes about 2 minutes 20 seconds.

If you use zc.buildout, then you can try the branch by checking it out,
adding a "develop" entry into your buildout config referencing it, and
updating any version spec for zope.testing to "3.6dev".  I'd really like
third-party confirmation of the total test time reductions I've seen.
--
Benji York
Senior Software Engineer
Zope Corporation


More information about the Zope-Dev mailing list