jakarta-cactus-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Hawtin <thaw...@tackline.demon.co.uk>
Subject Testing without reflection
Date Sat, 15 Mar 2003 16:03:38 GMT
A month ago I was looking to write some tests to run under MIDP. MIDP 
doesn't have reflection, and even if it did it wouldn't be a good idea 
to use it. Similarly reflection is not available for tests run in 
applets and servers with security set. To get around it is fiddly. So 
having nothing better to do, I wrote a little tool to generate static 
code to replace the reflection.

I'm not sure where to go with it, or even if it is really that useful. I 
think probably it is best becoming part of something else, for instance 
Cactus. Opinions?

I've put the text of the introduction below. The source is at:

    http://www.tackline.demon.co.uk/java/jpillar/jpillar.zip

Tom Hawtin




JPillar

JPillar permits JUnit-style test cases to be run in environments with 
security enabled or where the reflection API is not available. It 
becomes possible to easily run tests on production configured web 
servers, browsers and mobiles. Note that this is very much a 
pre-pre-alpha and only something I'm doing for 'fun' - it even lacks its 
own tests!

This product includes software developed by the Apache Software 
Foundation (http://www.apache.org/). Specifically the files 
ant/uk/co/demon/tackline/jpillar/JPillarTask.java and 
compiler/org/apache/bcel/ClassRepository.java are based upon files from 
Ant 1.5.1 and BCEL 5.0 respectively. See those files for more 
information and full copyright.
How it works

The JPillar tool compiles byte code to run test case methods without 
using reflection. Reflection can be a problem because some J2ME 
platforms do not support it and the security policies of some 
applications do not permit it. Using JPillar tests can be run in 
situations closer to their deployed environment and without coding 
overhead. Both J2MEUnit and JUnit are supported. Other styles should be 
relatively easy to add (see the Framework interface).

For each class that matches the relevant frameworks prerequists for a 
test, a class is created to run the tests. For instance, the standard 
JUnit interpretation matches any class ending in "Test". These are then 
treated as if they extend TestCase (there's no particular reason why 
they should extend or implement any particular type) and a test instance 
for each test method is produced. The generated class has a suffix of 
"TestSuite". A new instance is a test that tests all of the test methods.

Unfortunately the JUnit runners have code that throws secuirty 
exceptions. Worse the code is very fond of statics and wont load due to 
a static initialiser failing. At the very least in order to use it, 
change junit.runner.BaseTestRunner.getPreferecnes such that the secuirty 
exception from readPreferences does not propagate.

     try {
         readPreferences();
     } catch (java.lang.SecurityException exc) {
         System.err.println("Not permitted to read preferences file.");
     }

Writing tests

Note: the exact configuration is subject to change. For instance I might 
introduce marker types for test cases. See Framework implementations to 
see the current specifications.
JUnit test

Existing tests should run as usual. New JPillar test cases can be 
simplified.

Firstly, you do not need to extend TestCase - the generated class 
implements Test. Having said that extending Assert (TestCase's 
superclass) will usually be necessary. Typical exceptions to the rule 
are: using Java 1.5 (use: import static junit.framework.Assert;) or have 
your own assert class.

Secondly there is no need for main or suite methods. These are handled 
by the generated class. In fact there is no need to go through hoops to 
invoke the suite method as instantiating the class through the no-args 
constructor does the same.

There is a slight caveat in that setUp and tearDown if present need to 
be accessible at package level or empty (this is checked). Also you will 
need a no-args constructor (probably the default).
J2MEUnit test

Existing tests should run as usual. New JPillar test cases can be 
simplified. The simplification is more extreme than JUnit as the 
J2MEUnit tests start off more complicated.

As JUnit you do not need to extend TestCase, or have the main and suite 
methods. In J2MEUnit the suite method is high maintenance. Even more so 
is the runTest method which can also be dispensed with.

The same caveats setUp, tearDown and a no-arg constructor applies as JUnit.
Running the JPillar compiler
Command Line Interface

The compiler can be run from the command line with the syntax:
java -d dir [-framework (junit | j2meunit)] -classpath path [-run] 
[-verify] classfiles ...

-d dir - Sets the directory to write files. Mandatory.

-framework (junit | j2meunit) - Determines which framework to generate 
code for. junit is the default.

-classpath path - Is the entire classpath necessary for the tests. It 
should include your classes, junit.jar and rt.jar (or j2meunit and 
midpapi.zip). Mandatory.

-run - Runs the generated code.

-verify - Verifies the generated code.
ANT task

The task requires to be defined before it can be run

     &lt;taskdef name="jpillar"
          classname="uk.co.demon.tackline.jpillar.JPillarTask"
          classpath="${jpillar}/jpillartask.jar"
     /&gt;

The task is similar to javac, requiring srcdir, destdir and classpath. 
The framework can also be set to either "junit" or "j2meunit" with the 
former as the default. For instance:

      &lt;jpillar framework="junit"
          srcdir="${classes}"
          destdir="${testclasses}"
          classpath="${classes}:%{junit}:${rtjar}"&gt;
       &lt;patternset includes="**/*.class"/&gt;
     &lt;/jpillar&gt;

As a servlet filter

To make it easier to run tests in applets a SuiteFilter generates 
classes on the fly. Not sure a filter is the best approach, as it can't 
obviously be extended to generate a TestPackage class (I suppose a 
servlet for generating directory listings would plug the gap).
Running JPillar classes
 From the JPillar compiler

Just add -run to the command line. The tests are run in a class loader 
context separate from the compiler, but within the same process.

Command Line Interface

Simply run the generated class. There is no need for you to provide your 
own main method in your test case. The runner used is textui, but can 
easily be changed (see the Framework class).
java -classpath . pkg.MyTestSuite
Through a JUnit runner

The generated class provides a suite method, so again can be used as if 
it was a classic JUnit TestCase. For instance:
java -classpath junit.jar junit.textui.TestRunner pkg.MyTestSuite
As an applet

Two applets are provided. Both require the slightly modified JUnit (see 
above). AppletRunner (see etc/webapp/appletrunner.html) runs the test 
with the textui and dumps the results to a text area. JAppletRunner (see 
etc/webapp/jappletrunner.html) runs the test with the swingui. Note the 
swingui has reduced functionality and because of its somewhat excentric 
approach to multithreading, do not resize the appletviewer window while 
it is starting. An applet parameter "class" determines which class is run.
 From a servlet

The WebRunner servlet runs the test through the (modified) JUnit textui, 
returning the results in the page. Use a URL such as: 
http://localhost:8080/jpillar/webrun?class=uk.co.demon.tackline.jpillar.runtime.junit.SampleTestSuite
Through a Midlet

The classes supplied with J2MEUnit make reference to a non-MIDP/CLDC 
class, so you'll need to recompile them (JPillar build.xml does that). 
Also the WTK 2.0 beta I have doesn't like J2MEUnit for some reason, 
although 1.04 is fine. Unfortunately MIDP 1.0 only supports per JAD user 
attributes, not per Midlet as MIDP 2.0. 
uk.co.demon.tackline.jpillar.runtime.j2meunit.AttributeRunner, although 
rough and ready, will run a comma separated list of TestCase class 
names. Note my build.xml breaks lines in the JAD which need to be fixed 
(I shouldn't be using the manifest task).

Other ways

An EJB runner should be trivial. Applets and indeed midlets ought to be 
able to POST their results back. Should only be a small change to Cactus 
(or a filter), to make it run generated versions of the tests on the 
server side (whilst retaining reflection mechanism for beginXxx/endXxx 
methods on the client. Really ought to be using the Ant 
JUnit/JUnitReport tasks and Cactus mechanisms for creating reports.
Easing TestCase main without JPillar

It is possible to avoid writing a custom main, even without the use of 
JPillar. You still need a main, but crucially it does not need to refer 
to the name of the class itself. Use the main method public static void 
main(String[] args) { main(new Error()); } and extend the class below. 
You can add this to whatever base test case class you use for your 
project. This code is 1.4 or later only, but you can see how JUnit reads 
a stack trace in order to do it for earlier versions.

public abstract class TestCaseMain extends junit.framework.TestCase {
     protected static void main(Throwable traceable) {
        try {
            junit.textui.TestRunner.run(Class.forName(
                    traceable.getStackTrace()[0].getClassName()
            ));
            System.exit(0);
        } catch (Throwable exc) {
            exc.printStackTrace();
            System.exit(1);
        }
     }
}

And Bob becomes your uncle. For the perverts out there, use a static 
initialiser, temporarily redirect System.err, and start a thread that 
re-enables System.err and runs the test case (this will happen after the 
rest of the static inialisation of your class).
Building

In order to perform a full build JPillar, you require the following

     * Apache Ant - to perform the build and to build the Ant task. It 
is also used by the command line compiler (although I guess those 
classes will move across to Commons IO?).
     * Jakarta BCEL (Byte Code Engineering Library) - basic requirement. 
Note I have added a class (ClassRepository) in the supplied source.
     * JUnit - for the JUnit specific runtime parts. To make the small 
modification to enable it to run with security, you will also need the 
source.
     * J2ME - for the J2MEUnit specific runtime parts.
     * The Servlet API - for instance tomcat.

I fully admit that the build.xml is a complete mess. But it's the sort 
of excrutiatingly dull task I hate.
Stuff to do/think about
Subclassing versus delegation

I have decided to use delegation rather than subclassing for the 
relationship between the generated class and the target test case. There 
is a disadvantage in that protected methods in superclasses from 
different packages become inaccessible. To guard against this being a 
stealth issue, JPillar checks that if the test case setUp/tearDown 
methods are inaccessible, then they should only contain the single 
instruction RETURN.

My main reason for choosing delegation is that it just feels better the 
subtyping. An instance of the generated class is used as a collection of 
instances that map onto a single test method. This would be unpleasant 
if the generated class was a subclass of the test case class. Delegation 
also allows a decoupling between the framework used by the test case, 
and that used by the runner. If you like, you can also make the 
generated class extend, say, your own Midlet.
Miscellaneous

     * Tests!
     * Cactus integration
     * Nicer and more runners
     * Run tests on different system to presentation (Cactus style).
     * User configurable Framework
     * Sort out build.xml
     * Web JUnit test collector - so swingui, for instance, can give a 
choice of tests to run.
     * Better documentation!
     * Mutable security manager with asserts for use of permissions.
     * Static java.lang.reflect.Proxy replacement
     * Generate source and debugging for generated classes
     * Create TestPackage/AllTests classes.

End note

I only wrote this because I was writing a Go midlet (with server stuff) 
and didn't like the smell of J2ME unit and playing with BCEL looked fun. 
I haven't approached this project with my professional standards, and 
have got pissed off with fiddling about with random pieces of software. 
For instance, it would make vastly more sense to have written a 
prototype using in-class-loader-context reflection to generate Java 
source files. Would a tidied version be of much use to anyone? I can't 
really answer that. Perhaps integrated as part of say, Cactus, the cost 
of entry would be sufficiently low.



Mime
View raw message