[Zope3-checkins]
SVN: zope.testing/trunk/src/zope/testing/testrunner
Show the total time to run all tests.
Marius Gedminas
marius at pov.lt
Sun Jul 15 08:03:41 EDT 2007
Log message for revision 77993:
Show the total time to run all tests.
Changed:
U zope.testing/trunk/src/zope/testing/testrunner-colors.txt
U zope.testing/trunk/src/zope/testing/testrunner-coverage.txt
U zope.testing/trunk/src/zope/testing/testrunner-edge-cases.txt
U zope.testing/trunk/src/zope/testing/testrunner-errors.txt
U zope.testing/trunk/src/zope/testing/testrunner-layers-api.txt
U zope.testing/trunk/src/zope/testing/testrunner-layers-ntd.txt
U zope.testing/trunk/src/zope/testing/testrunner-layers.txt
U zope.testing/trunk/src/zope/testing/testrunner-leaks.txt
U zope.testing/trunk/src/zope/testing/testrunner-profiling-cprofiler.txt
U zope.testing/trunk/src/zope/testing/testrunner-profiling.txt
U zope.testing/trunk/src/zope/testing/testrunner-repeat.txt
U zope.testing/trunk/src/zope/testing/testrunner-simple.txt
U zope.testing/trunk/src/zope/testing/testrunner-wo-source.txt
U zope.testing/trunk/src/zope/testing/testrunner.py
-=-
Modified: zope.testing/trunk/src/zope/testing/testrunner-colors.txt
===================================================================
--- zope.testing/trunk/src/zope/testing/testrunner-colors.txt 2007-07-15 11:42:50 UTC (rev 77992)
+++ zope.testing/trunk/src/zope/testing/testrunner-colors.txt 2007-07-15 12:03:40 UTC (rev 77993)
@@ -163,7 +163,7 @@
Tear down samplelayers.Layer122 in 0.000 seconds.
Tear down samplelayers.Layer12 in 0.000 seconds.
Tear down samplelayers.Layer1 in 0.000 seconds.
- {normal}Total: {green}413{normal} tests, {boldred}3{normal} failures, {boldred}1{normal} errors{normal}
+ {normal}Total: {green}413{normal} tests, {boldred}3{normal} failures, {boldred}1{normal} errors in {green}0.023{normal} seconds.{normal}
True
The expected and actual outputs of failed doctests are shown in different
Modified: zope.testing/trunk/src/zope/testing/testrunner-coverage.txt
===================================================================
--- zope.testing/trunk/src/zope/testing/testrunner-coverage.txt 2007-07-15 11:42:50 UTC (rev 77992)
+++ zope.testing/trunk/src/zope/testing/testrunner-coverage.txt 2007-07-15 12:03:40 UTC (rev 77993)
@@ -49,7 +49,7 @@
Tear down samplelayers.Layer122 in 0.000 seconds.
Tear down samplelayers.Layer12 in 0.000 seconds.
Tear down samplelayers.Layer1 in 0.000 seconds.
- Total: 405 tests, 0 failures, 0 errors
+ Total: 405 tests, 0 failures, 0 errors in N.NNN seconds.
lines cov% module (path)
...
52 92% sample1.sampletests.test1 (testrunner-ex/sample1/sampletests/test1.py)
Modified: zope.testing/trunk/src/zope/testing/testrunner-edge-cases.txt
===================================================================
--- zope.testing/trunk/src/zope/testing/testrunner-edge-cases.txt 2007-07-15 11:42:50 UTC (rev 77992)
+++ zope.testing/trunk/src/zope/testing/testrunner-edge-cases.txt 2007-07-15 12:03:40 UTC (rev 77993)
@@ -429,7 +429,7 @@
TypeError: Invalid test_suite, None, in sample1.sampletests_none_suite
<BLANKLINE>
<BLANKLINE>
- Total: 0 tests, 0 failures, 0 errors
+ Total: 0 tests, 0 failures, 0 errors in N.NNN seconds.
<BLANKLINE>
Test-modules with import problems:
sample1.sampletests_none_suite
@@ -447,7 +447,7 @@
TypeError: ...
<BLANKLINE>
<BLANKLINE>
- Total: 0 tests, 0 failures, 0 errors
+ Total: 0 tests, 0 failures, 0 errors in N.NNN seconds.
<BLANKLINE>
Test-modules with import problems:
sample1.sampletests_none_test
Modified: zope.testing/trunk/src/zope/testing/testrunner-errors.txt
===================================================================
--- zope.testing/trunk/src/zope/testing/testrunner-errors.txt 2007-07-15 11:42:50 UTC (rev 77992)
+++ zope.testing/trunk/src/zope/testing/testrunner-errors.txt 2007-07-15 12:03:40 UTC (rev 77993)
@@ -123,7 +123,7 @@
Tear down samplelayers.Layer122 in 0.000 seconds.
Tear down samplelayers.Layer12 in 0.000 seconds.
Tear down samplelayers.Layer1 in 0.000 seconds.
- Total: 413 tests, 3 failures, 1 errors
+ Total: 413 tests, 3 failures, 1 errors in N.NNN seconds.
True
We see that we get an error report and a traceback for the failing
@@ -885,7 +885,7 @@
Tear down samplelayers.Layer122 in 0.000 seconds.
Tear down samplelayers.Layer12 in 0.000 seconds.
Tear down samplelayers.Layer1 in 0.000 seconds.
- Total: 213 tests, 0 failures, 0 errors
+ Total: 213 tests, 0 failures, 0 errors in N.NNN seconds.
<BLANKLINE>
Test-modules with import problems:
sample2.sampletests_i
@@ -970,7 +970,7 @@
... # doctest: +ELLIPSIS
Running unit tests:
...
- Total: 364 tests, 0 failures, 0 errors
+ Total: 364 tests, 0 failures, 0 errors in N.NNN seconds.
...
sys.exit was not called
Modified: zope.testing/trunk/src/zope/testing/testrunner-layers-api.txt
===================================================================
--- zope.testing/trunk/src/zope/testing/testrunner-layers-api.txt 2007-07-15 11:42:50 UTC (rev 77992)
+++ zope.testing/trunk/src/zope/testing/testrunner-layers-api.txt 2007-07-15 12:03:40 UTC (rev 77993)
@@ -118,7 +118,7 @@
Ran 2 tests with 0 failures and 0 errors in N.NNN seconds.
Tearing down left over layers:
Tear down BaseLayer in N.NNN seconds.
-Total: 4 tests, 0 failures, 0 errors
+Total: 4 tests, 0 failures, 0 errors in N.NNN seconds.
Now lets specify a layer in the suite containing TestSpecifyingNoLayer
and run the tests again. This demonstrates the other method of specifying
@@ -150,7 +150,7 @@
Tearing down left over layers:
Tear down TopLayer in N.NNN seconds.
Tear down BaseLayer in N.NNN seconds.
-Total: 4 tests, 0 failures, 0 errors
+Total: 4 tests, 0 failures, 0 errors in N.NNN seconds.
If we inspect our trace of what methods got called in what order, we can
see that the layer setup and teardown methods only got called once. We can
Modified: zope.testing/trunk/src/zope/testing/testrunner-layers-ntd.txt
===================================================================
--- zope.testing/trunk/src/zope/testing/testrunner-layers-ntd.txt 2007-07-15 11:42:50 UTC (rev 77992)
+++ zope.testing/trunk/src/zope/testing/testrunner-layers-ntd.txt 2007-07-15 12:03:40 UTC (rev 77993)
@@ -74,7 +74,7 @@
<BLANKLINE>
Ran 6 tests with 2 failures and 2 errors in N.NNN seconds.
Tear down sample3.sampletests_ntd.Layer ... not supported
- Total: 8 tests, 2 failures, 2 errors
+ Total: 8 tests, 2 failures, 2 errors in N.NNN seconds.
True
in the example above, some of the tests run as a subprocess had errors
@@ -152,7 +152,7 @@
<BLANKLINE>
Ran 6 tests with 0 failures and 4 errors in N.NNN seconds.
Tear down sample3.sampletests_ntd.Layer ... not supported
- Total: 8 tests, 0 failures, 4 errors
+ Total: 8 tests, 0 failures, 4 errors in N.NNN seconds.
True
Similarly, pdb.set_trace doesn't work when running tests in a layer
@@ -232,7 +232,7 @@
<BLANKLINE>
Ran 7 tests with 0 failures and 0 errors in 0.008 seconds.
Tear down sample2.sampletests_ntds.Layer ... not supported
- Total: 8 tests, 0 failures, 0 errors
+ Total: 8 tests, 0 failures, 0 errors in N.NNN seconds.
False
If you want to use pdb from a test in a layer that is run as a
Modified: zope.testing/trunk/src/zope/testing/testrunner-layers.txt
===================================================================
--- zope.testing/trunk/src/zope/testing/testrunner-layers.txt 2007-07-15 11:42:50 UTC (rev 77992)
+++ zope.testing/trunk/src/zope/testing/testrunner-layers.txt 2007-07-15 12:03:40 UTC (rev 77993)
@@ -26,7 +26,7 @@
Tear down samplelayers.Layerx in N.NNN seconds.
Tear down samplelayers.Layer11 in N.NNN seconds.
Tear down samplelayers.Layer1 in N.NNN seconds.
- Total: 226 tests, 0 failures, 0 errors
+ Total: 226 tests, 0 failures, 0 errors in N.NNN seconds.
False
We can also specify that we want to run only the unit tests:
@@ -72,7 +72,7 @@
Tear down samplelayers.Layer122 in N.NNN seconds.
Tear down samplelayers.Layer12 in N.NNN seconds.
Tear down samplelayers.Layer1 in N.NNN seconds.
- Total: 213 tests, 0 failures, 0 errors
+ Total: 213 tests, 0 failures, 0 errors in N.NNN seconds.
False
Or we can explicitly say that we want both unit and non-unit tests.
@@ -112,6 +112,6 @@
Tear down samplelayers.Layer122 in N.NNN seconds.
Tear down samplelayers.Layer12 in N.NNN seconds.
Tear down samplelayers.Layer1 in N.NNN seconds.
- Total: 405 tests, 0 failures, 0 errors
+ Total: 405 tests, 0 failures, 0 errors in N.NNN seconds.
False
Modified: zope.testing/trunk/src/zope/testing/testrunner-leaks.txt
===================================================================
--- zope.testing/trunk/src/zope/testing/testrunner-leaks.txt 2007-07-15 11:42:50 UTC (rev 77992)
+++ zope.testing/trunk/src/zope/testing/testrunner-leaks.txt 2007-07-15 12:03:40 UTC (rev 77993)
@@ -48,7 +48,7 @@
Tearing down left over layers:
Tear down samplelayers.Layer12 in 0.000 seconds.
Tear down samplelayers.Layer1 in 0.000 seconds.
- Total: 68 tests, 0 failures, 0 errors
+ Total: 68 tests, 0 failures, 0 errors in N.NNN seconds.
Each layer is repeated the requested number of times. For each
iteration after the first, the system refcount and change in system
Modified: zope.testing/trunk/src/zope/testing/testrunner-profiling-cprofiler.txt
===================================================================
--- zope.testing/trunk/src/zope/testing/testrunner-profiling-cprofiler.txt 2007-07-15 11:42:50 UTC (rev 77992)
+++ zope.testing/trunk/src/zope/testing/testrunner-profiling-cprofiler.txt 2007-07-15 12:03:40 UTC (rev 77993)
@@ -25,7 +25,7 @@
...
Running samplelayers.Layer11 tests:
...
- Total: ... tests, 0 failures, 0 errors
+ Total: ... tests, 0 failures, 0 errors in ... seconds.
...
ncalls tottime percall cumtime percall filename:lineno(function)...
Modified: zope.testing/trunk/src/zope/testing/testrunner-profiling.txt
===================================================================
--- zope.testing/trunk/src/zope/testing/testrunner-profiling.txt 2007-07-15 11:42:50 UTC (rev 77992)
+++ zope.testing/trunk/src/zope/testing/testrunner-profiling.txt 2007-07-15 12:03:40 UTC (rev 77993)
@@ -25,7 +25,7 @@
...
Running samplelayers.Layer11 tests:
...
- Total: ... tests, 0 failures, 0 errors
+ Total: ... tests, 0 failures, 0 errors in ... seconds.
...
ncalls tottime percall cumtime percall filename:lineno(function)...
Modified: zope.testing/trunk/src/zope/testing/testrunner-repeat.txt
===================================================================
--- zope.testing/trunk/src/zope/testing/testrunner-repeat.txt 2007-07-15 11:42:50 UTC (rev 77992)
+++ zope.testing/trunk/src/zope/testing/testrunner-repeat.txt 2007-07-15 12:03:40 UTC (rev 77993)
@@ -38,7 +38,7 @@
Tear down samplelayers.Layerx in 0.000 seconds.
Tear down samplelayers.Layer11 in 0.000 seconds.
Tear down samplelayers.Layer1 in 0.000 seconds.
- Total: 226 tests, 0 failures, 0 errors
+ Total: 226 tests, 0 failures, 0 errors in N.NNN seconds.
False
The tests are repeated by layer. Layers are set up and torn down only
Modified: zope.testing/trunk/src/zope/testing/testrunner-simple.txt
===================================================================
--- zope.testing/trunk/src/zope/testing/testrunner-simple.txt 2007-07-15 11:42:50 UTC (rev 77992)
+++ zope.testing/trunk/src/zope/testing/testrunner-simple.txt 2007-07-15 12:03:40 UTC (rev 77993)
@@ -80,7 +80,7 @@
Tear down samplelayers.Layer122 in N.NNN seconds.
Tear down samplelayers.Layer12 in N.NNN seconds.
Tear down samplelayers.Layer1 in N.NNN seconds.
- Total: 405 tests, 0 failures, 0 errors
+ Total: 405 tests, 0 failures, 0 errors in N.NNN seconds.
False
we see the normal testrunner output, which summarizes the tests run for
Modified: zope.testing/trunk/src/zope/testing/testrunner-wo-source.txt
===================================================================
--- zope.testing/trunk/src/zope/testing/testrunner-wo-source.txt 2007-07-15 11:42:50 UTC (rev 77992)
+++ zope.testing/trunk/src/zope/testing/testrunner-wo-source.txt 2007-07-15 12:03:40 UTC (rev 77993)
@@ -64,7 +64,7 @@
... os.remove(os.path.join(root, f))
>>> testrunner.run(mydefaults, ["test", "--keepbytecode"])
Running tests at level 1
- Total: 0 tests, 0 failures, 0 errors
+ Total: 0 tests, 0 failures, 0 errors in N.NNN seconds.
False
Finally, passing ``--usecompiled`` asks the test runner to treat .pyc
Modified: zope.testing/trunk/src/zope/testing/testrunner.py
===================================================================
--- zope.testing/trunk/src/zope/testing/testrunner.py 2007-07-15 11:42:50 UTC (rev 77992)
+++ zope.testing/trunk/src/zope/testing/testrunner.py 2007-07-15 12:03:40 UTC (rev 77993)
@@ -380,10 +380,10 @@
print (" Ran %s tests with %s failures and %s errors in %.3f seconds."
% (n_tests, n_failures, n_errors, n_seconds))
- def totals(self, n_tests, n_failures, n_errors):
+ def totals(self, n_tests, n_failures, n_errors, n_seconds):
"""Summarize the results of all layers."""
- print "Total: %s tests, %s failures, %s errors" % (
- n_tests, n_failures, n_errors)
+ print "Total: %s tests, %s failures, %s errors in %.3f seconds." % (
+ n_tests, n_failures, n_errors, n_seconds)
def list_of_tests(self, tests, layer_name):
"""Report a list of test names."""
@@ -698,7 +698,7 @@
self.color('info'), ' seconds.',
self.color('normal'), '\n'])
- def totals(self, n_tests, n_failures, n_errors):
+ def totals(self, n_tests, n_failures, n_errors, n_seconds):
"""Report totals (number of tests, failures, and errors)."""
sys.stdout.writelines([
self.color('info'), 'Total: ',
@@ -707,7 +707,9 @@
self.error_count_color(n_failures), str(n_failures),
self.color('info'), ' failures, ',
self.error_count_color(n_errors), str(n_errors),
- self.color('info'), ' errors',
+ self.color('info'), ' errors in ',
+ self.color('number'), '%.3f' % n_seconds,
+ self.color('info'), ' seconds.',
self.color('normal'), '\n'])
def print_traceback(self, msg, exc_info):
@@ -1034,6 +1036,8 @@
output.list_of_tests(tests, layer_name)
return True
+ start_time = time.time()
+
for layer_name, layer, tests in layers_to_run:
nlayers += 1
try:
@@ -1051,6 +1055,8 @@
output.info("Tearing down left over layers:")
tear_down_unneeded(options, (), setup_layers, True)
+ total_time = time.time() - start_time
+
if options.resume_layer:
sys.stdout.close()
# Communicate with the parent. The protocol is obvious:
@@ -1066,7 +1072,7 @@
output.tests_with_failures(failures)
if nlayers != 1:
- output.totals(ran, len(failures), len(errors))
+ output.totals(ran, len(failures), len(errors), total_time)
output.modules_with_import_problems(import_errors)
More information about the Zope3-Checkins
mailing list