[Grok-dev] Re: where do I put doctests for my models?

[Uli Fouquet]

Hi Brandon,

Am Dienstag, den 07.08.2007, 10:08 -0400 schrieb Brandon Craig Rhodes:
> Brandon Craig Rhodes <brandon at rhodesmill.org> writes:
> 
> > I have written a Grok Model and now need to know where to put its
> > doctest so that it gets run when I type "./bin/test". ...
> 
> I have done some more work on my question; PvW's book was helpful.
> The answer to my question seems to be that Grok does not currently
> give any help to the developer who wants testing - that there exists
> almost no "convention", but almost entirely "configuration".

Well, you got at least out-of-the-box a testrunner as defined in
bin/test and a functional test layer (ftesting.zcml). I am really not an
expert in such things, but as far as I can see there is in fact nothing
Grok-specific concerning tests. It's all plain Zope 3, isn't it?

I was wondering, whether it would be useful for beginners to have some
dumb testing modules already available as boilerplates when they install
a fresh Grok project.

On the one hand this would increase the number of files of a 'stock Grok
project' and might confuse beginners. On the other hand it could
encourage use of tests, which is, what we all want (I neglected this at
the beginning of my project; a really miserable mistake ;-).

> Could I submit a tutorial showing the current state-of-the-art in
> building tests for a Grok app, so that other beginners have somewhere
> to start from?

After your first posting I started to write a mini-tutorial for
automated testing with Grok. It turned out to quickly become something
quite complex, so it might be a 'maxi'-tutorial.

The first 'chapter' is finished, but lots of important topics are still
missing. Therefore I didn't publish it yet. However, you might have some
use for it and want to extend it. You can find the source here:

http://svn.zope.org/Sandbox/ulif/grok-adminui/doc/minitutorials/testing.txt

and the rendered version here:

https://gf1.gnufix.de/~uli/grok/www/minitutorials/testing.html

Because I am currently a bit busy with other issues, I won't extend it
today but you are very welcome to do a collaborative effort to finish
the text :-)

> To get a doctest to run in my Grok buildout when I type "./bin/test",
> the following seems necessary:
> 
>  - I must create a directory called "tests" in my app.
>  - I must create an __init__.py in that directory.
>  - I must create files with names like test*.py in the directory.
>  - Those files must have functions or classes in them named test_*.
>  - Those functions or classes must "from doctest import DocFileSuite"
>    and then list, filename by filename, all of my doctest files; or
>    I have to write my own recursive-descent routine (the grok tests
>    themselves seem to do this) to search for tests in my app.
> 
> I humbly suggest that all of the above is a massive example of
> configuration over convention - the programmer has to perform about
> six steps, each of which must be done perfectly correctly, before the
> "./bin/test" command becomes willing to run his tests.
> 
> Would it be possible for the "test" command in a Grok buildout to
> automatically search for tests, instead of making the developer write
> his own "test_app.py" module?  I would love to be able to drop dozens
> of ".doctest" files in my "tests" directory inside my app, and have
> them just magically get run when I type "./bin/test".

I'd place doctests into the source itself or txt-files, such avoiding
another filename extension. It's fairly simple to scan a package for
such files and do an automated testing out-of-the-box. However, I am not
sure, whether this would be too much black magic.

> I am willing to help code this, but am not sure what should change -
> the code of "./bin/test" itself, or something else?

Not sure, but IMHO the code, that decides where to look for tests should
remain in the application package, i.e. below the src-directory. Putting
it in a subdirectory 'src/projectname/tests', like it is done in the
grok package, seems like a reasonable convention to me, well understood
also by non-zopistas.

Making the testrunner to scan exotic things from my point of view would
be too much of black magic and confuse people easily.

Just my 2c.

Cheers,

-- 
Uli