commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Luc Maisonobe <Luc.Maison...@free.fr>
Subject Re: [MATH] Documenting test failures
Date Sat, 05 Sep 2009 18:38:51 GMT
Phil Steitz a écrit :
> sebb wrote:
>> There are quite a lot of assertions that don't give any details if a test fails.
>>
>> For example, the following currently fails when using Harmony:
>>
>> testIncreasingTolerance(org.apache.commons.math.ode.nonstiff.AdamsBashforthIntegratorTest)
>> java.lang.AssertionError:
>> 	at org.junit.Assert.fail(Assert.java:74)
>> 	at org.junit.Assert.assertTrue(Assert.java:37)
>> 	at org.junit.Assert.assertTrue(Assert.java:46)
>> 	at org.apache.commons.math.ode.nonstiff.AdamsBashforthIntegratorTest.testIncreasingTolerance(AdamsBashforthIntegratorTest.java:89)
>>
>> Obviously one can find the code from the line number, but that does
>> not actually get one much further as there's no indication what the
>> values are:
>>
>>             assertTrue(handler.getMaximalValueError() < (36.0 *
>> scalAbsoluteTolerance));
>>
>> Seems to me it might be useful to create some additional assert
>> methods, for example:
>>
>> assertLessThan([message,] maximum, actual)
>>
>> which would report the max and actual values if the check failed.
>>
>> As far as I can tell, this is not currently supported by JUnit3/4 -
>> only assertSame/assertEquals report the expected and actual values.
>>
>> WDYT?
>>
>> There are also lots of cases of assert() calls without a message;
>> again this means digging into the test code to find the problem.
>>
>> I propose to add messages to at least the ones that Harmony triggers. OK?
> 
> I am fine with adding messages, as long as they are not misleading.

I'm not a big fan of messages in Junit tests. These tests are mainly
used in two successive contexts:

  - first during code development (sometimes in a test driven process),
 - second during long terme maintainance

In the first case the tests may fail but the developer already knows
exactly the purpose of each test. In the second case the tests should
never fail and when they do people always have to look at both the test
code and library code, so the message is merely a convenience. What
bothers me is that these messages are a pain to maintain and sometimes
as new tests are added by copying an existing test and changing a few
things, the messages are often not changed in the process. Last thing is
that due to lazyness sometimes we get things like:

  assertEquals("a != b", a, b);

which clearly is not informative.

So I would say that if you want to add such tests, you can do it but I
think this will be a huge task for small improvements.

Luc

> 
> Phil
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
> 
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org


Mime
View raw message