3.6.0 of zope.testing is out with my subprocess parallelization branch merged (plus I fixed a few bugs in the trunk). There were several other changes, here's the pertinent part of the release notes:
- Added -j option to parallel tests run in subprocesses.
- RENormalizer accepts plain Python callables.
- Added --slow-test option.
- Added --no-progress and --auto-progress options.
- Complete refactoring of the test runner into multiple code files and a more modular (pipeline-like) architecture.
- Unified unit tests with the layer support by introducing a real unit test layer.
- Added a doctest for ``zope.testing.module``. There were several bugs that were fixed:
* ``README.txt`` was a really bad default argument for the module name, as it is not a proper dotted name. The code would immediately fail as it would look for the ``txt`` module in the ``README`` package. The default is now ``__main__``.
* The tearDown function did not clean up the ``__name__`` entry in the global dictionary.
* Fix a bug that caused a SubprocessError to be generated if a subprocess sent any output to stderr.
* Fix a bug that caused the unit tests to be skipped if run in a subprocess.
Hey,
Cool, thanks for the release. I notice by the way that my clever use of nested lists confused two bug fixers; only the first two entries here have to do with my fixes in zope.testing.module. :)
Benji York wrote: [snip]
Added a doctest for ``zope.testing.module``. There were several bugs that were fixed:
``README.txt`` was a really bad default argument for the module name, as it is not a proper dotted name. The code would immediately fail as it would look for the ``txt`` module in the ``README`` package. The default is now ``__main__``.
The tearDown function did not clean up the ``__name__`` entry in the global dictionary.
Fix a bug that caused a SubprocessError to be generated if a subprocess sent any output to stderr.
Fix a bug that caused the unit tests to be skipped if run in a subprocess.
Regards,
Martijn
On Fri, Jul 11, 2008 at 6:02 AM, Martijn Faassen faassen@startifact.com wrote:
Hey,
Cool, thanks for the release. I notice by the way that my clever use of nested lists confused two bug fixers; only the first two entries here have to do with my fixes in zope.testing.module. :)
Now fixed on the trunk.
Hey Benji,
Seems that it breaks coverage...
The result looks like coverage is started after importing the modules to test. That means declarations do not seem to 'run', but just the code inbetween them. Checked with 3.5.1, it looks fine.
On Wed, Jul 16, 2008 at 1:19 PM, Adam GROSZER agroszer@gmail.com wrote:
Hey Benji,
Seems that it breaks coverage...
In what way is coverage broken? With or without -j? Can you formulate a test that demonstrates the breakage?
The result looks like coverage is started after importing the modules to test. That means declarations do not seem to 'run', but just the code inbetween them. Checked with 3.5.1, it looks fine.
I suspect the breakage happened prior in the lifetime of 3.6.0 to my changes. If you would, please try to find the revision that introduced the breakage. My first comparison would be between r85079 and r86460.
Hello Benji,
Sorry, now being specific with the facts (all other activities only tomorrow (I'll check the revisions))
Results are generated by: python bootstrap.py bin/buildout bin/coverage-test bin/coverage-report
on the source of: svn://svn.zope.org/repos/main/Sandbox/adamg/ocql/branches/optimize-with-index (WARNING: buildout.cfg has now zope.testing nailed to 3.5.1) same python, same everything, just zope.testing switched
Good coverage is: (with 3.5.1): lines cov% module (path) 40 100% ocql.aoptimizer.aoptimizer (/home/adi/ocql/src/ocql/aoptimizer/aoptimizer.py) 8 87% ocql.aoptimizer.tests (/home/adi/ocql/src/ocql/aoptimizer/tests.py) 135 85% ocql.compiler.compiler (/home/adi/ocql/src/ocql/compiler/compiler.py) 45 60% ocql.compiler.runnablequery (/home/adi/ocql/src/ocql/compiler/runnablequery.py) 10 90% ocql.compiler.tests (/home/adi/ocql/src/ocql/compiler/tests.py) 31 93% ocql.database.index (/home/adi/ocql/src/ocql/database/index.py) 69 94% ocql.database.metadata (/home/adi/ocql/src/ocql/database/metadata.py) 8 87% ocql.database.tests (/home/adi/ocql/src/ocql/database/tests.py) 29 100% ocql.engine (/home/adi/ocql/src/ocql/engine.py) 35 100% ocql.interfaces (/home/adi/ocql/src/ocql/interfaces.py) 239 72% ocql.parser.queryparser (/home/adi/ocql/src/ocql/parser/queryparser.py) 9 88% ocql.parser.tests (/home/adi/ocql/src/ocql/parser/tests.py) 19 100% ocql.qoptimizer.qoptimizer (/home/adi/ocql/src/ocql/qoptimizer/qoptimizer.py) 9 88% ocql.qoptimizer.tests (/home/adi/ocql/src/ocql/qoptimizer/tests.py) 326 83% ocql.queryobject.queryobject (/home/adi/ocql/src/ocql/queryobject/queryobject.py) 150 88% ocql.rewriter.algebra (/home/adi/ocql/src/ocql/rewriter/algebra.py) 54 100% ocql.rewriter.interfaces (/home/adi/ocql/src/ocql/rewriter/interfaces.py) 18 100% ocql.rewriter.rewriter (/home/adi/ocql/src/ocql/rewriter/rewriter.py) 9 88% ocql.rewriter.tests (/home/adi/ocql/src/ocql/rewriter/tests.py) 62 90% ocql.testing.database (/home/adi/ocql/src/ocql/testing/database.py) 28 100% ocql.testing.sample.interfaces (/home/adi/ocql/src/ocql/testing/sample/interfaces.py) 9 88% ocql.testing.sample.mentor (/home/adi/ocql/src/ocql/testing/sample/mentor.py) 9 88% ocql.testing.sample.organization (/home/adi/ocql/src/ocql/testing/sample/organization.py) 9 88% ocql.testing.sample.project (/home/adi/ocql/src/ocql/testing/sample/project.py) 10 100% ocql.testing.sample.student (/home/adi/ocql/src/ocql/testing/sample/student.py) 73 100% ocql.testing.stubs (/home/adi/ocql/src/ocql/testing/stubs.py) 80 100% ocql.testing.utils (/home/adi/ocql/src/ocql/testing/utils.py) 85 96% ocql.testing.utils_opt (/home/adi/ocql/src/ocql/testing/utils_opt.py) 250 94% ocql.tests.test_old (/home/adi/ocql/src/ocql/tests/test_old.py) 17 94% ocql.tests.test_skeleton (/home/adi/ocql/src/ocql/tests/test_skeleton.py) 15 93% ocql.tests.test_utils (/home/adi/ocql/src/ocql/tests/test_utils.py) 105 88% ocql.tests.test_zope (/home/adi/ocql/src/ocql/tests/test_zope.py)
Bad coverage is (with 3.6.0): lines cov% module (path) 39 53% ocql.aoptimizer.aoptimizer (/home/adi/ocql/src/ocql/aoptimizer/aoptimizer.py) 134 30% ocql.compiler.compiler (/home/adi/ocql/src/ocql/compiler/compiler.py) 44 22% ocql.compiler.runnablequery (/home/adi/ocql/src/ocql/compiler/runnablequery.py) 29 24% ocql.database.index (/home/adi/ocql/src/ocql/database/index.py) 68 50% ocql.database.metadata (/home/adi/ocql/src/ocql/database/metadata.py) 28 35% ocql.engine (/home/adi/ocql/src/ocql/engine.py) 238 26% ocql.parser.queryparser (/home/adi/ocql/src/ocql/parser/queryparser.py) 18 27% ocql.qoptimizer.qoptimizer (/home/adi/ocql/src/ocql/qoptimizer/qoptimizer.py) 325 41% ocql.queryobject.queryobject (/home/adi/ocql/src/ocql/queryobject/queryobject.py) 149 39% ocql.rewriter.algebra (/home/adi/ocql/src/ocql/rewriter/algebra.py) 17 29% ocql.rewriter.rewriter (/home/adi/ocql/src/ocql/rewriter/rewriter.py) 61 8% ocql.testing.database (/home/adi/ocql/src/ocql/testing/database.py) 9 11% ocql.testing.sample.student (/home/adi/ocql/src/ocql/testing/sample/student.py) 72 25% ocql.testing.stubs (/home/adi/ocql/src/ocql/testing/stubs.py) 79 72% ocql.testing.utils (/home/adi/ocql/src/ocql/testing/utils.py) 85 96% ocql.testing.utils_opt (/home/adi/ocql/src/ocql/testing/utils_opt.py) 249 79% ocql.tests.test_old (/home/adi/ocql/src/ocql/tests/test_old.py) 16 25% ocql.tests.test_skeleton (/home/adi/ocql/src/ocql/tests/test_skeleton.py) 14 21% ocql.tests.test_utils (/home/adi/ocql/src/ocql/tests/test_utils.py) 104 60% ocql.tests.test_zope (/home/adi/ocql/src/ocql/tests/test_zope.py)
Sample bad result:
from ocql.interfaces import IObjectQuery from ocql.interfaces import IOptimizedObjectQuery from ocql.interfaces import IAlgebraObject from ocql.interfaces import IOptimizedAlgebraObject from ocql.interfaces import IRunnableQuery
class OCQLEngine: implements(IEngine)
def __init__(self):
9: pass
def compile(self, query):
#TODO: later use maybe named adapters 26: metadata = IDB(None)
26: if IObjectQuery.providedBy(query): 21: objectquery = query else: 5: objectquery = IQueryParser(query)(metadata) 26: optimizedoq = IQueryOptimizer(objectquery)()
Wednesday, July 16, 2008, 7:29:29 PM, you wrote:
BY> On Wed, Jul 16, 2008 at 1:19 PM, Adam GROSZER agroszer@gmail.com wrote:
Hey Benji,
Seems that it breaks coverage...
BY> In what way is coverage broken? With or without -j? Can you formulate BY> a test that demonstrates the breakage?
The result looks like coverage is started after importing the modules to test. That means declarations do not seem to 'run', but just the code inbetween them. Checked with 3.5.1, it looks fine.
BY> I suspect the breakage happened prior in the lifetime of 3.6.0 to my BY> changes. If you would, please try to find the revision that introduced BY> the breakage. My first comparison would be between r85079 and r86460.
Hello,
Seems like 86460 breaks it.
I have some idea why testrunner-coverage.txt does not detect this. I think it is doing coverage just on the tests code. There seems to be no "application"-like code. All code seems to come from testcases, doctests, docfiles. So far I can see.
Right now I'm under some time pressure so implementation (of the test) will have to wait.
On Thu, Jul 17, 2008 at 3:20 AM, Adam GROSZER agroszer@gmail.com wrote:
Hello,
Seems like 86460 breaks it.
I have some idea why testrunner-coverage.txt does not detect this. I think it is doing coverage just on the tests code. There seems to be no "application"-like code. All code seems to come from testcases, doctests, docfiles. So far I can see.
Right now I'm under some time pressure so implementation (of the test) will have to wait.
Any sign of pressure relief?
Hello Benji,
I think we solved that problem at the Blackforest sprint in Freiburg with Roger. As I remember there are tests for that too.
Wednesday, October 22, 2008, 8:46:19 PM, you wrote:
BY> On Thu, Jul 17, 2008 at 3:20 AM, Adam GROSZER agroszer@gmail.com wrote:
Hello,
Seems like 86460 breaks it.
I have some idea why testrunner-coverage.txt does not detect this. I think it is doing coverage just on the tests code. There seems to be no "application"-like code. All code seems to come from testcases, doctests, docfiles. So far I can see.
Right now I'm under some time pressure so implementation (of the test) will have to wait.
BY> Any sign of pressure relief?
Hi Benji, Adam
Betreff: Re: [Zope-dev] zope.testing 3.6.0 released
Hello Benji,
I think we solved that problem at the Blackforest sprint in Freiburg with Roger. As I remember there are tests for that too.
Yes, we solved the issue that the coverage feature was loaded after the import feature. This was ending in none covered import lines and global module variable (application-like code).
This was fixed in zope.testing revision 90220: - let the coverage feature start earlier then the find feature - adjust test coverage test
and should be released in 3.5.5
Regards Roger Ineichen
Wednesday, October 22, 2008, 8:46:19 PM, you wrote:
BY> On Thu, Jul 17, 2008 at 3:20 AM, Adam GROSZER agroszer@gmail.com wrote:
Hello,
Seems like 86460 breaks it.
I have some idea why testrunner-coverage.txt does not detect this. I think it is doing coverage just on the tests code. There
seems to
be no "application"-like code. All code seems to come from
testcases,
doctests, docfiles. So far I can see.
Right now I'm under some time pressure so implementation (of the test) will have to wait.
BY> Any sign of pressure relief?
-- Best regards, Adam GROSZER mailto:agroszer@gmail.com -- Quote of the day: I am always doing that which I can not do, in order that I may learn how to do it.
- Pablo Picasso
Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
-
Adam GROSZER -
Benji York -
Martijn Faassen -
Roger Ineichen