[Zope-CVS] CVS: Packages/FunctionalTests/doc - CHANGES.txt:1.1 FunctionalTests.odm:1.1 ISSUES.txt:1.1 README.txt:1.1

Tres Seaver tseaver@zope.com
Thu, 24 Apr 2003 17:39:54 -0400


Update of /cvs-repository/Packages/FunctionalTests/doc
In directory cvs.zope.org:/tmp/cvs-serv3971/doc

Added Files:
	CHANGES.txt FunctionalTests.odm ISSUES.txt README.txt 
Log Message:


  - Repackage for distutils.


=== Added File Packages/FunctionalTests/doc/CHANGES.txt ===
FunctionalTests Changelog

  Release 0.4 -- 2003/04/24

    - Packaged for distutils.

  Release 0.3 -- 2002/04/30

    - Framework: Modified Result to propagate cookies through entire
      result tree.

    - Framework: added 'repeat' directive for both requests and sub-tests.

    - Framework: added SleepRequest, to permit injection of "shim" time
      between real requests.

  Release 0.2 -- 2002/03/14

    - FTRunner:  added parsing of arbitrary defaults from command-line
      as name=value pairs.

    - Framework: added ability to capture and return server-originated
      cookies (e.g., to allow testing of session-based applications).

  Release 0.1 -- 2002/01/??

    - Initial release


=== Added File Packages/FunctionalTests/doc/FunctionalTests.odm === (2427/2527 lines abridged)
ObjectDomain_Model 10
{z8kflt_-bfo_cs5fga3q_-p98 0
}[objectdomain.uml.behavioral.collaboration.Collaboration-owns $19
objectdomain.uml.core.TypeExpression $31
objectdomain.uml.behavioral.common.ReturnAction-actionContainer $25
objectdomain.uml.core.UmlTaggedValue-taggedValues $12
objectdomain.uml.view.ClassifierRoleView-owns $8
objectdomain.sg.sgraphics.SGImage $34
objectdomain.uml.core.UmlOperation-features $15
objectdomain.uml.mechanisms.UmlModel-models $1
objectdomain.uml.core.UmlAttribute-features $17
objectdomain.uml.mechanisms.StereotypePackage-owns $26
objectdomain.domain.DomainLockObject $33
objectdomain.uml.behavioral.common.CallAction-actionContainer $23
objectdomain.domain.ModelEnvironment-owns $2
objectdomain.uml.behavioral.common.UmlArgument-arguments $24
objectdomain.uml.view.GeneralizationView-owns $5
objectdomain.uml.behavioral.collaboration.ClassifierRole-owns $20
objectdomain.uml.view.Diagram-owns $3
objectdomain.uml.mechanisms.StylePackage-owns $28
objectdomain.uml.mechanisms.UmlPackage-owns $10
objectdomain.uml.view.MessageView-owns $9
objectdomain.uml.core.UmlExpression $32
objectdomain.uml.core.UmlParameter-parameters $16
objectdomain.uml.core.Projection $29
objectdomain.uml.core.UmlAssociationEnd-connections $14
objectdomain.uml.core.UmlMultiplicity $30
objectdomain.uml.core.Generalization-owns $18
objectdomain.uml.view.AssociationView-owns $6
objectdomain.uml.view.ClassView-owns $4
objectdomain.uml.core.UmlClass-owns $11
objectdomain.uml.core.UmlAssociation-owns $13
objectdomain.uml.behavioral.common.CreateAction-actionContainer $22
objectdomain.uml.core.Stereotype-owns $27
objectdomain.uml.behavioral.collaboration.UmlMessage-owns $21
objectdomain.uml.view.SequenceDiagram-sequenceDiagrams $7
]
($1 1 > z8kflt_-bfo_cs5fga3q_-p98 0
(name "Model"
)
(uid "z8kflt_-bfo_cs5fga3q_-p98")
(namespaceRoot true))
($2 2 < z8kflt_-bfo_cs5fga3q_-p97 "ModelEnvironment" 1
(uid "z8kflt_-bfo_cs5fga3q_-p97"))
($10 47 < z8kflt_-bfo_cs5fga3q_-p7o "Logical" 1
(name "Logical"
)
(uid "z8kflt_-bfo_cs5fga3q_-p7o")
(namespaceRoot true))
($3 3 < z8kflt_-bfo_cs5fga3q_-p7n "Logical diagram" 47

[-=- -=- -=- 2427 lines omitted -=- -=- -=-]

(receiverAssn 357 356)
(successorAssn 357 362)
(senderAssn 357 353)
(projectionAssn 36 357 473)
(to 37 32)
(from 37 31)
(projectionAssn 37 359 476)
(to 38 33)
(from 38 32)
(projectionAssn 38 362 477)
(projectionAssn 39 365 478)
(to 40 34)
(from 40 33)
(projectionAssn 40 366 479)
(to 41 39)
(from 41 34)
(projectionAssn 41 370 480)
(to 42 39)
(from 42 34)
(projectionAssn 42 372 481)
(to 43 35)
(from 43 34)
(projectionAssn 43 379 482)
(to 44 39)
(from 44 34)
(projectionAssn 44 384 483)
(to 45 39)
(from 45 34)
(projectionAssn 45 387 484)
(to 46 31)
(from 46 32)
(projectionAssn 46 390 485)
(openDiagrams 2 30)
(selection 2 11)
(lockedBy 47 486)
(icon 394 487)
(lockedBy 394 486)
(stereotypeAssn 47 394)
(icon 432 488)
(icon 433 489)
(icon 434 490)
(lockedBy 393 486)
(lockedBy 436 486)
(lockedBy 437 486)
(stereotypeAssn 437 394)
(lockedBy 438 486)
(stereotypeAssn 438 394)
(lockedBy 439 486)
(stereotypeAssn 439 394)
(lockedBy 1 486)

=== Added File Packages/FunctionalTests/doc/ISSUES.txt ===
Issues

  - "Local" configuration (hostname, port #, site root) is mixed up
    with "general" stuff (request definition).

  - Running the driver requires having the environment set up "just
    right" (correct Python, correct INSTANCE_HOME/SOFTWARE_HOME/PYTHONPATH,
    etc.)

  - Ideally, the tests would run against an isolated Zope instance,
    which served from a DemoStorage, backed by a FileStorage in a known
    state (lots of the setup code would go away here).

  - Could we "record" scenarios for later replay, perhap using John
    Odom's derived HTTPPublisher?  Alternately, could we parse the
    scripts generated by OpenSTA?


=== Added File Packages/FunctionalTests/doc/README.txt ===
FunctionTests README

  What is this?

    FunctionalTests is a set of utilities for building automated
    functional tests of web applications.  Such tests are useful
    for several reasons:

    - They provide a crisp specification of the
      externally-visible "web API" of the application.

    - They make delivered functionality visible to the customer
      in an objective fashion:  if the customer believes that the
      tests are sufficient, and the tests all pass, then we are
      done.

    - They allow for simple instrumentation of the application,
      chiefly to permit developers to identify and correct
      software errors which lead to grossly-unacceptable scaling
      or performance.

    - They provide a fairly realistic basis for doing load
      testing of a site, as the sequence of requests which
      comprise a use case are, by definition, likely to occur
      "together" in typical use.

  How does it work?

    The package structure of functional tests for an application
    should follow closely the package structure of its use cases.
    Thus, each "use case package" in the system should map to a
    "functional testing package", which is a directory, possibly
    within a hierarchy.

    In an incrementally-delivered application (such as the
    typical consulting gig), each increment will have a package
    of use cases which it implements, and a corresponding package
    of functional tests for the use cases.

    Each use case should have one or more functional tests which
    exercise it.

    Each functional test consists of a set of request
    specifications, which if executed successfully (and in order)
    demonstrate that the use case is satisfied.  In particular,
    the functional test specification contains:
    
      o Specification of any preconditions / setup needed to run
        the test, expressed as a Python Script / External
        Method.  The directory for the functional test should
        contain the source of this method.

      o Specification of the request sequence which fulfills
        the use case, including:

        + URL

        + Authentication

        + Any other cookies or HTTP headers

        + POST data, if any

        + PUT content, if any

      o Specification of a postcondition for the case again
        expressed as a Python Script or External Method;   this
        method should return an HTTP error response code if the
        postcondition is not met.  As with the setup method, the
        source for the postcondition method should be stored in
        the directory containing the functional test.

      o Specification of any cleanup required at the close of
        the request, expressed as a Python Script or External
        Method.  Again, the source for this method should be
        stored in the directory with the functional test.

  Sample Usage

    Within the Python debugger::

     >>> defaults = { 'portal_url':'http://localhost:8080/functest'
     ...            , 'site_path':'/functest' }
     >>> from FunctionalTests.Framework import buildTest
     >>> t = buildTest( '/home/tseaver/projects/ZopeOrg'
     ...                '/functional_tests/CMF_base/submit_news.txt'
     ...              , defaults=defaults )
     >>> r = t( app=app, defaults=defaults )
     >>> assert r()
     >>> for invocation in r.listInvocations():
     ...     print invocation.getReplyStatus()
     ...
     200 OK
     200 OK
     200 OK
     200 OK
     200 OK
     302 Moved Temporarily
     302 Moved Temporarily
     200 OK
     302 Moved Temporarily
     200 OK
     200 OK
     302 Moved Temporarily


  File Formats

    Packages

      For the product being tested, create a 'functional_tests'
      subdirectory (like the one this file is in).  It should
      consist of a README (like this one) and one or more
      subdirectories for the various increments (at least one
      should contain "baseline" functional tests for Zope and
      CMF).

      For each increment, create a new subdirectory.  Add a README
      file, describing the general scope and goals of the
      increment, and enumerating the use cases (qualifying as
      needed) which it is to implement.

    Package Specification Files

      Each package (directory) in the functional testing
      hierarchy must specify its contents using an "INI-file"
      like file (parseable by the standard Python ConfigParser
      module).  This file consists of a pair of sections:

      - '[Package]' contains metadata about the package, and
        enumerates the tests and sub-packages which it contains.

      - '[DEFAULT]' contains a set of standard substitution
        values, which are propagated to each test and
        sub-package as global defaults.

      Example package specification file::

        [Package]
        Title: Browsing the site
        Status: Active
        Tests: browse_news, browse_events
        Packages: Folder_contents

        [DEFAULT]
        portal_url: http://localhost:8080/functest

    Scenario Specification Files

      For each scenario of a tested use case, create an "INI-file"
      like file (one parseable by the standard Python
      ConfigParser).  The '[Scenario]' section should contain the
      title of the scenario, and the name of its use case.

      Each request section should contain fields for the URL,
      the authentication tokens, a list of form fields, a
      list of cookies, and a list of additional headers.

      Example scenario specification::

        [Scenario]
        Title: Browse news
        Use_case: Guest explores site.

        [Setup]
        # adds '/testnews', plus two news items, and makes them published
        URL: %(portal_url)s/scaffolding/CMF_base/browse_news_setup

        [Home_Page]
        URL: %(portal_url)s/
        Expected_Result: 200

        [News_Page]
        URL: %(portal_url)s/recent_news
        Expected_Result: 200

        [News_Item_1]
        URL: %(portal_url)s/testnews/item_1
        Expected_Result: 200

        [News_Item_2]
        URL: %(portal_url)s/testnews/item_2
        Expected_Result: 200

        [Teardown]
        # removes '/testnews'.
        URL: %(portal_url)s/scaffolding/CMF_base/browse_news_teardown