commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gilles Sadowski <gil...@harfang.homelinux.org>
Subject Re: [Math] Gump failure for "BaseSecantSolverTest"
Date Mon, 27 Jun 2011 09:42:37 GMT
On Mon, Jun 27, 2011 at 12:09:29AM +0100, sebb wrote:
> On 26 June 2011 17:58, Gilles Sadowski <gilles@harfang.homelinux.org> wrote:
> >>
> >> Some tools are smart enough to detect it, but not all of them.
> >>
> >> >At least, maven's conclusion is correct: It knows that there no tests to
run
> >> >(or rather that it should not try to instantiate an abstract class) despite
> >> >the fact that there are methods marked with the "@Test" annotation.
> >>
> >> Yes, but it find it after having failed to launch these tests. It
> >> does not check it beforehand.
> >
> > How do you see that?
> > When I run "mvn test" here, the output does not show "BaseSecantSolverTest"
> > at all, and the build is successful. [Otherwise I wouldn't have committed
> > that class.]
> 
> Yes, because Surefire detects that the class is abstract.

Thus, do you know whether this tool could be used from ant too?

> Did you try "ant test" ?

Not before committing the offending class, if this what you ask.
I usually run "mvn clean site", look at the output of the tests, and at the
CheckStyle-generated HTML page.

> JUnit does not detect it until it tries to run it.

Thus Junit does not analyse the code as Surefire does, but there is an
annotation ("@Ignore") to tell it not to run any tests within the annotated
class. As a side-effect, it does not try to instantiate it.

> >> >
> >> >If it's possible, it is certainly more robust to rely on a Java keyword
> >> >("abstract") rather than on an external convention (a name ending with
> >> >"AbstractTest"). [Or, more precisely, even if the name does not end with
> >> >"AbstracTest", the information exists that can prevent generating an error.]
> >>
> >> It would be a good thing, I didn't check if there was a feature
> >> request on the tools we use for this (ant, maven, surefire, gump,
> >> continuum, eclipse ...).
> >>
> >> >The introduction of the "@Test" annotation is a progress, going in that
> >> >direction; that is, it is not mandatory anymore that the name of a test
> >> >method starts with "test". Similarly, it should not be mandatory that a
> >> >base class for tests ends with "AbstractTest".
> >>
> >> I don't know all the annotations Junit uses. Perhaps there is
> >> something for the complete class.
> >
> > I'll check whether instanitation could be disabled at that level; that would
> > indeed be the best option.
> 
> No. the best option is to use the proper naming convention, so the
> tools don't even try to run the class as a test.

It used to be the only option because the tools were not smart enough. Now
it is not necessary anymore (in the same way that test methods do _have_ to
start with "test").
Before it was necessary, now it's a matter of taste.

> >> >
> >> >>>Maven also relies
> >> >>>on the surefire plugin to run the tests, and surefire relies on
Junit. So
> >> >>>there are a lot of intermediate steps between a build system (Gump,
> >> >>>Continuum, direct use of ant, direct use of Maven, Eclipse, Eclipse
with
> >> >>>maven plugin ...) and the low level Junit runs. This may explain
the
> >> >>>differences.
> >> >>
> >> >>Indeed.
> >> >>
> >> >>>I have already noticed that many tools do not have the same algorithm
to
> >> >>>select classes to test. A recent example was the performance tests
for
> >> >>>FastMath. Maven skip these tests because they do not end in "Test",
but
> >> >>>Eclipse for example does not skip them because it directly look
inside the
> >> >>>class and find the @Test annotations.
> >> >>
> >> >>[Yes, that Eclipse behaviour is a nuisance!]
> >> >>
> >> >>It *could* have been that the Ant build file and Maven Pom had
> >> >>different configs for the test file names.  However, I've just
> >> >>checked, and the files agree.
> >> >>
> >> >>The difference is due to the way that Surefire currently processes
> >> >>test-classes before handing them to JUnit.
> >> >
> >> >Do you mean that it's Surefire that handles the things correctly?
> >> >Then, could it be used also by Gump?
> >>
> >> It does not handle it correctly, it still tries to launch the tests
> >> and when it fails, there is an error message.
> >
> > Then I don't understand why it works locally, but not on the build server...
> > [No error message here.]
> 
> Because you are using Maven, not Ant.

Well, I started use maven especially for building commons-math because I was
advised to do so, ant being not well supported (at the time). Did it change?

By the way, I noticed that, in order to run a single test case, the
property definition is not the same with ant and maven:
  ant -Dtest.entry=org.apache.commons.math.analysis.solvers.SomeTest test
vs.
  mvn -Dtest=SomeTest test

[With ant, the fully-qualified name of the class is required.]

> >> >
> >> >>>There is clearly no ideal solution, but I think having different
build
> >> >>>systems to suit several users needs is better. Having different
tools help
> >> >>>finding different bugs.
> >> >>
> >> >>Indeed - in this case, the wrong naming convention for an abstract test
class.
> >> >
> >> >If it's impossible to properly configure Gump, I'll change the name...
> >>
> >> I think this is the better short term solution.
> >>
> >
> > Probably, if there is no Junit annotation.
> 
> Using the correct naming convention is the solution.

It's not _the_ correct solution, it's one solution; using "@Ignore" is now
another which arguably is more robust since the behaviour will be the same,
whatever tool calls Junit, without relying on out-of-band conventions or
intermediate tools.


Regards,
Gilles

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org


Mime
View raw message