lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: failure in TestTrieRangeQuery
Date Wed, 04 Feb 2009 13:55:58 GMT

Great!!

Yes please open JIRA.

Mike

Uwe Schindler wrote:

> Hi,
>
> atached is a patch for LuceneTestCase and a implementation in
> TestTrieRangeQuery. It overrides the protected method runTest() and  
> inserts
> a try-catch around the super.runTest() call. Two new methods  
> newRandom() and
> newRandom(long) are available for the test case. As each test case  
> is run in
> an own TestCase object instance (so 5 test methods in a class  
> instantiate 5
> instances each method working in separate), the random seed is saved  
> on
> newRandom() and when the test fails with any Throwable, a message  
> with the
> seed (if not null) is printed out. If newRandom was never called no  
> message
> will be printed.
>
> This patch has only one problem: If a single test method calls  
> newRandom()
> more than once, only the last seed is saved and printed out. But  
> each test
> method in a Testcase should call newRandom() exactly once for usage  
> during
> the execution of this test method. And it is not thread save (no  
> sync, no
> volatile), but for tests it's unimportant.
>
> Maybe we open a JIRA issue?
>
> Uwe
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>> -----Original Message-----
>> From: Michael McCandless [mailto:lucene@mikemccandless.com]
>> Sent: Wednesday, February 04, 2009 12:02 PM
>> To: java-dev@lucene.apache.org
>> Subject: Re: failure in TestTrieRangeQuery
>>
>>
>> Sweet!
>>
>> I was wondering (but didn't dig) whether we could extend
>> LuceneTestCase to expose a getRandom() method (which'd record the
>> seed), and then override invocation of a test (which I'm not sure
>> JUnit allows you to do) to add a try/finally that prints out the  
>> seeds.
>>
>> Though: I thought JUnit invokes tests in the sequential order as they
>> are defined in your class?  (I'm not sure about this... it's just  
>> what
>> seems to be the case).  And even if it is the case, it's not clear
>> that's guaranteed as part of JUnit's "contract".  If it isn't, we
>> could have getRandom take a String name and then on exception we  
>> print
>> out the full name -> seed for all getRandom calls for that test?
>>
>> I'd like to to find a simple common API, if we can, so that we can  
>> fix
>> all tests that use Random to use it... though really the mods you had
>> to make are fairly minimal, so we could simply adopt that per test  
>> too.
>>
>> Mike
>>
>> Uwe Schindler wrote:
>>
>>> Hi,
>>>
>>>>> : By allowing Random to randomly seed itself, we effectively  
>>>>> test a
>>>>> much
>>>>> : much larger space, ie every time we all run the test, it's
>>>>> different.  We can
>>>>> : potentially cast a much larger net than a fixed seed.
>>>>>
>>>>> i guess i'm just in favor of less randomness and more iterations.
>>>>>
>>>>> : Fixing the bug is the "easy" part; discovering a bug is  
>>>>> present is
>>>>> where
>>>>> : we need all the help we can get ;)
>>>>>
>>>>> yes, but knowing a bug is there w/o having any idea what it is or
>>>>> how to
>>>>> trigger it can be very frustrating.
>>>>
>>>> I agree, it's frustrating.  But I'd prefer to know the bug is there
>>>> and then
>>>> writhe in frustration at not being able to reproduce it very  
>>>> easily,
>>>> then let
>>>> the bug go undetected.  I guess ignorance is not bliss, for me ;)
>>>>
>>>>> it would be enough for tests to pick a random number, log it, and
>>>>> then use
>>>>> it as the seed ... that way if you get a failure you at least know
>>>>> what
>>>>> seed was used and you can then hardcode it temporarily to  
>>>>> reproduce/
>>>>> debug
>>>>
>>>> +1!  I like this approach.  We could record the seed up front, and
>>>> then in
>>>> a try/finally if the test failed, print the seed.
>>>
>>> I implemented this for TestTrieRangeQuery (see patch). I catch
>>> java.lang.Error and print the random seed recorded before (The  
>>> seed is
>>> generated by a static Random instance for each test method in
>>> separate:
>>> Because you cannot predict the order of tests, each test method
>>> should have
>>> its own Random instance). As both the Java 1.4 AssertionError and
>>> the jUnit
>>> AssertionFailedError are subclasses of Error, they can be catched  
>>> and
>>> rethrown easily.
>>>
>>> Uwe
>>> <random-trie-
>>> test
>>> .patch
>>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
> <LuceneTestCase- 
> random 
> .patch 
> >---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message