[Zope-Coders] Regular Zope tests

[Martijn Pieters]

On Mon, Oct 22, 2001 at 05:14:26PM -0400, Matthew T. Kromer wrote:
> See incentive above.  The point here is not to sugar coat the test 
> suites.  If you would prefer, I'll generate the test mailing instead.  I 
> started doing that earlier but never hooked up a mailer.  
> 
> As a developer, you might only care if things are broken.  As a 
> packager, I want to know when things are good.  I want someone to tell 
> me, daily, that things are good.  I'm amenable to being flexible about 
> unpublished code (e.g. the trunk) but I still think its more likely that 
> bad test cases get fixed if they're announced frequently, rather than 
> announced once and then set to a "broken and forgotten" state.

Sounds like we need a web and a mail interface, like the Mozilla tinderbox:

  http://tinderbox.mozilla.org/showbuilds.cgi?tree=SeaMonkey

It'll show, on any given platform:

- a timeline, with links to CVS checkins.

- Build in progress in yellow

- Source compilation failures in red.

- Test errors in orange.

- links to build and test output

- ability to add comments to builds, so others can see what is being done
  about breakage.

Obviously, we don't need anything this elaborate with multiple build
platforms, but the combination of color coding and checkin messages gives
you a very good overview of the state of the tree.

(Just in case you are wondering what Clbr and Dep mean: Clbr stands for
Clobber, run a 'make clean' before CVS up and compile, while Dep builds
without a make clean. Thus, Clbr builds take longer, and are run on more
powerful boxes).

Anyway, the web interface will give the packager a clear view of the health
of the tree, and only failures in the tests will be mailed, so developers
immediately know if a test broke.

-- 
Martijn Pieters
| Software Engineer  mailto:mj@zope.com
| Zope Corporation   http://www.zope.com/
| Creators of Zope   http://www.zope.org/
---------------------------------------------