[Zope-Coders] Regular Zope tests

[Matthew T. Kromer]

Shane Hathaway wrote:

> Matthew T. Kromer wrote:
>
>> Shane Hathaway wrote:
>>
>>> I got excited about the number of unit tests Zope now passes (937 
>>> out of 939), so I decided to document it by beginning automatic 
>>> posting of test results to zope-coders@zope.org.  It will post at 10 
>>> PM on weekdays, but only when any test results change.
>>>
>>> Enjoy!
>>>
>> I assert that all failed cases should post nightly, or nights prior 
>> to work days (ie Sunday - Thursday).
>
>
>
> Right now no one is assigned to fix the last two test cases.  Maybe 
> once they are fixed we should switch to always mailing failures.


And if you never report failures, there is a lower incentive to assign 
people to fix them.  The point is to have it be a nusiance.  I'd even 
endorse a "so-and-so broke the build" mailing if there was a change 
attributable to a person.

>> An OK test run should affirmatively mail that NNN/NNN test cases 
>> successfully ran.  I'm OK with the thing shutting up after some 
>> number of days that everything is good, but I would still like to see 
>> weekly reminders that things are good (which also means that the 
>> auto-tester is good).
>
> Steve suggested that we might also provide a web interface.  I think 
> that would fulfill the need to find out the current state of things.


The point is to be a daily affirmation of the state of the tree.  If I 
wanted to know if the tests ran succesfully whenever-I-was-curious, I 
could pull my own tree and run the tests.

>> I'd like to see test case mailings for both the current stable and 
>> development branches.
>
>
> Currently the branches fail a *lot* of the tests, mainly just because 
> the test scripts need to be updated.  Once they are sorted out, we can 
> auto-test them too.
>
> Shane


See incentive above.  The point here is not to sugar coat the test 
suites.  If you would prefer, I'll generate the test mailing instead.  I 
started doing that earlier but never hooked up a mailer.  

As a developer, you might only care if things are broken.  As a 
packager, I want to know when things are good.  I want someone to tell 
me, daily, that things are good.  I'm amenable to being flexible about 
unpublished code (e.g. the trunk) but I still think its more likely that 
bad test cases get fixed if they're announced frequently, rather than 
announced once and then set to a "broken and forgotten" state.