ant-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@apache.org>
Subject <junit> and junit4 (long)
Date Thu, 25 May 2006 12:36:55 GMT
Jesse Glick and I had a phone conf with a couple of JUnit people, 
including Kent Beck, on the future of <junit>

They've been getting reports of junit4/<junit> problems, which is not 
surprising given that ant1.65. doesn't work with junit4. We discussed 
what is coming in Ant1.7, and I guess junit4 is extra motivation for us 
to stick a beta of that out soon.

We discussed the recurrent support calls related to <junit> that we see

  -junit.jar not on ant's classpath

  -setting up the xml listener (needs xml parser on the path)

  -xml reports don’t fully escape everything they should, and when the app
   terminates there is an empty xml file (because it buffers everything 
in a test case)

  -<junit> tries to run abstract test cases (no check for the abstract bit)

  -people don't get their selection patterns right, and we try and run 
things that arent tests. We could do a check here for 3.x, but not for 
junit 4.

  -People get extra burned when their pattern is **/Test* and they have 
a nested inner class, because of course that TestSomething$Inner.class 
matches the pattern

  -not well integrated with logging APIs. we get the text as printed to 
stdout, but lose the full log metadata. If we had something behind 
commons-logging we could capture logs as data rather than as stdout 
messages.

  -most of the methods are public, we know the task is delegated to, and 
probably
   subclassed, so are constrained in what we can change.

Jesse also explained what is done for junit4 support
  -exception class name pattern matching to distinguish faults from errors
  -suite() support
  -reflection for all this stuff so it builds on java1.4
  -picking up of junit4 before junit3

The consensus was that a new junit4 antlib/task would be the way 
forward, one that had the same attributes/elements as <junit>, better 
defaults (assertions on, fork=true?), and designed to run junit4 (and so 
compile on java5 only).
It could then add new stuff for junit4, like recognise ignored tests and 
still report them. We'd need a new task/stylesheet for reporting junit4 
runs.

We discussed where this task would go (antlib under ant vs. antlib at 
junit.org), and what the stability/redist implications would be. junit 
uses java which is very backwards compatible, though ant1.7 improves 
some things. Putting it in with junit4 gives it a tighter coupling to 
junit.

I think the consensus was that the junit CVS repository (on sourceforge) 
is the place to do this, but that ant developers need to provide help, 
ant users can provide input on wants/needs.

1. Who wants to help work on a new junit task?

2. What should it do?

3. Are there any enhancements we'd like to see in junit4 classes to help 
us?


My responses

(1) I would, with that caveat about no free time. I could send them the 
chapters
     of JdWA that cover writing ant tasks, when I finish them in the 
next two
     weeks (according to the schedule!)

(2) what should it do?

   -Everything in <junit>,
   -Offer control of any new options
   -Listener interface enhancements
    -allow for log events as well as stdout.
    -provide our own commons logging/java logging back ends that route to
     the listeners
    -set the forked jvm up with the right properties to route to these 
back ends

   -XML listener to be enhanced with
     -streaming of the output to file
     -stick the results at the end of each run, instead of attributes on 
the root
      node (this is implicit with streaming)


    -have some way of remembering which tests failed and only running 
them. This
     could be done with a listener that generates a file containing a 
list of
     classes that could be read by the task next time round and used to 
select
     classes to test

    -skip abstract tests always

    -skip inner classes (with a $ in the shortname) unless 
testInnerClasses=true
     (we could patch this back onto junit too)

3. enhancements to junit

    -a method "isFailure" on the assertion failure classes, for checking 
(through reflection in ant) for failures vs errors.

    -I'd like serializable forms of test results, fault data and 
statistics. I had to do my own for smartfrog junit components, code I 
now share with gridunit. Serialized classes make it easy to share 
results around or save
     them to a file.

There isnt a formal junit mail list for developing this task; I think 
there ought to be and then interested people can get on it. They are 
going to put their dev branch onto gump though.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org


Mime
View raw message