lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler (JIRA)" <j...@apache.org>
Subject [jira] Created: (LUCENE-1535) Make tests using java.util.Random reproducible on failure
Date Wed, 04 Feb 2009 14:09:59 GMT
Make tests using java.util.Random reproducible on failure
---------------------------------------------------------

                 Key: LUCENE-1535
                 URL: https://issues.apache.org/jira/browse/LUCENE-1535
             Project: Lucene - Java
          Issue Type: Test
    Affects Versions: 2.9
            Reporter: Uwe Schindler
             Fix For: 2.9


This is a patch for LuceneTestCase to support logging of the Random seed used in randomized
tests. The patch also includes an example implementation in TestTrieRangeQuery.

It overrides the protected method runTest() and inserts a try-catch around the super.runTest()
call. Two new methods newRandom() and newRandom(long) are available for the test case. As
each test case is run in an own TestCase object instance (so 5 test methods in a class instantiate
5 instances each method working in separate), the random seed is saved on newRandom() and
when the test fails with any Throwable, a message with the seed (if not null) is printed out.
If newRandom was never called no message will be printed.

This patch has only one problem: If a single test method calls newRandom() more than once,
only the last seed is saved and printed out. But each test method in a Testcase should call
newRandom() exactly once for usage during the execution of this test method. And it is not
thread save (no sync, no volatile), but for tests it's unimportant.

I forgot to mention: If a test fails, the message using the seed is printed to stdout. The
developer can then change the test temporarily:

{code}LuceneTestCase.newRandom() -> LuceneTestCase.newRandom(long){code}

using the seed from the failed test printout.

*Reference:*
{quote}
: By allowing Random to randomly seed itself, we effectively test a much
: much larger space, ie every time we all run the test, it's different.  We can
: potentially cast a much larger net than a fixed seed.

i guess i'm just in favor of less randomness and more iterations.

: Fixing the bug is the "easy" part; discovering a bug is present is where
: we need all the help we can get ;)

yes, but knowing a bug is there w/o having any idea what it is or how to 
trigger it can be very frustrating.

it would be enough for tests to pick a random number, log it, and then use 
it as the seed ... that way if you get a failure you at least know what 
seed was used and you can then hardcode it temporarily to reproduce/debug

-Hoss
{quote}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message