cocoon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Turner <j...@socialchange.net.au>
Subject Re: testing/performance framework for Cocoon
Date Sun, 23 Sep 2001 03:53:58 GMT
On Fri, Sep 21, 2001 at 08:33:13AM -0700, Ovidiu Predescu wrote:
> 
> Hi,
> 
> One of the most frustrating parts of developing Cocoon is the lack of
> a testsuite. Since I've started working with Cocoon, I've seen a
> feature working fine now, and completely broken in the next day's CVS
> snapshot. As a result I'm trying to come up with some framework to
> have automated regression testing, and maybe performance testing as
> well.
[..]
> The Ant based approach requires some extension tasks to be
> defined. Most of them seem to be easy to implement, only one of them
> is more difficult.
[..] 
> Below is a list of Ant extension tasks:
> 
> <database> - describe the database used to store the results.
> 
> <test> - defines a new test. Each test has a name, and whether the
> time taken to execute it should be logged in the database or not. The
> assumption is that some tests are really just that, while others are
> really performance measurements.
> 
> <url-get> - makes an HTTP GET request to an URL
> 
> <url-post> - makes an HTTP POST request to an URL

I recently required something like this. I first played around with Tomcat's
Ant task for doing unit testing (see
jakarta-tomcat-4.0/tester/src/bin/tester.xml). It's used like this:

<project name="Tester" default="all">
  <taskdef  name="tester"     classname="org.apache.tester.TestClient"/>
  ..
  <tester host="${host}" port="${port}" protocol="${protocol}"
           request="/index.html" debug="${debug}"
           status="200"/>
  <tester host="${host}" port="${port}" protocol="${protocol}"
         request="${context.path}/WrappedGolden01"
         golden="${golden.path}/Golden01.txt"/>
  <tester host="${host}" port="${port}" protocol="${protocol}"
          debug="${debug}"
          request="${context.path}/protected/Authentication01"
          inHeaders="Authorization:Basic dG9tY2F0OnRvbWNhdA=="
  ..

However, this system is rather monolithic and hard to extend. All new tests
would have to be added to TestClient directly.

I had a brief poke at httpunit, but it seemed to be more oriented at HTML
testing.

So I'm now playing with a testing tool called Latka, developed by Morgan
Delagrange in jakarta-commons/latka.

  "Latka is a functional (end-to-end) testing tool. It is implemented in Java,
  and uses an XML syntax to define a series of HTTP (or HTTPS) requests and a
  set of validations used to verify that the request was processed correctly."

  -- http://jakarta.apache.org/commons/latka/index.html

It has quite a few validation techniques already, but none specific to XML.
However, it's very easy to write new validators. I had a MIME type validator
(eg check for text/xml) and an XML validation validator going in a few hours. I
expect it would be quite trivial to implement an XPath test:

> <check> - checks an XPath in the XML document returned by an <url-get>
> or <url-post>.

Here's a sample XML file, doing a few tests against XML services:

<?xml version="1.0" standalone="no"?>
<!DOCTYPE suite SYSTEM "../conf/suite.dtd">
<suite defaultHost="newgate.socialchange.net.au" label="newgate">

    <request path="/sightings/servlet/WFSSightingsServlet" label="WFSSightings">
        <validate>
            <statusCode label="valid HTTP status code"  code="200"/>
            <mimeType label="Correct XML MIME type" value="text/xml"/>
        </validate>
    </request>

    <request host="clearinghouse1.fgdc.gov"
        path="/scripts/ogc/ms.pl?WMTVER=1.0&amp;request=CAPABILITIES"
        label="SCO WMS">
        <validate>
            <mimeType value="text/xml"/>
            <xmlValidator/>
        </validate>
    </request>
</suite>

(David Crossley and other GIS people will understand those tests;)

Note that it doesn't use Ant. I suspect this is actually a good thing. Ant 1.x
is not flexible enough to be easily repurposed as a http testing engine. This
will apparently be fixed in Ant 2, which will be a generic "task execution
engine", but for now, it's probably harder to turn Ant into something it's not,
than to start again. For instance, there have been *many* submissions of an
iterate, or <foreach> task:

> <iterate> - creates a "for" loop so we can do very simple loops

But the ant-dev team are very much against turning Ant into an XML scripting
language, and reject such tasks.

That thought leads to one serious suggestion: how about implementing the tests
in Python, and then using a Python-to-Java compiler
(http://jython.sourceforge.net/) to generate bytecode?

> <spawn> - creates a given number of threads and executes the tasks
> specified as children. This could be used to simulate multiple clients
> connecting to a URL.

It may be possible to hack Ant 1.4's <parallel> task to do this sort of thing.
However, the ant-dev committers would be unlikely to accept your addition,
since it amounts to an <iterate> task.

So I think the options (in order of my preference;) are:
 - use and extend Latka to do the testing (lots of work)
 - write tests in python, compile to java (less work)
 - turn Ant into testing tool with testing-specific tasks
 - use httpunit or another framework

--Jeff

[..]

> Greetings,
> -- 
> Ovidiu Predescu <ovidiu@cup.hp.com>
> http://orion.rgv.hp.com/ (inside HP's firewall only)
> http://sourceforge.net/users/ovidiu/ (my SourceForge page)
> http://www.geocities.com/SiliconValley/Monitor/7464/ (GNU, Emacs, other stuff)

---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org


Mime
View raw message