lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: failure in TestTrieRangeQuery
Date Tue, 03 Feb 2009 23:58:30 GMT
I used to favor determinism/repeatability (fixed seed to Random) too,
but I recently changed my mind.

By allowing Random to randomly seed itself, we effectively test a much
much larger space, ie every time we all run the test, it's different.   
We can
potentially cast a much larger net than a fixed seed.

This way, if there is a bug lurking, it will eventually be seen by
someone.

Once the bug is discovered (which is 90% of the "challenge"), we can
always run many iterations, test different seeds, etc., to track down
the cause & fix it.

Fixing the bug is the "easy" part; discovering a bug is present is where
we need all the help we can get ;)

Mike

Chris Hostetter wrote:

>
> : It's not repeatable, which is fine (because the test has  
> randomness, which we
> : should leave in there).
>
> Side note: while i agree that test with randomness (ie: do lots of
> iterations over randomly selected data) are good to help find weird  
> edge
> casees you might not otherwise think to explicitly code tests for,  
> i've
> found by painful experience that generating the random data using a  
> Random
> object with a hard coded seed is a good practice -- that way you get  
> the
> benefits of hte Random data, but the test is also reproducable.
>
> otherwise you get test failures that that you *think* you understand  
> the
> cause of based on the stack trace, but without knowing exactly what  
> input
> was used, you can't be sure you're fixing the "right" bug.
>
>
>
> -Hoss
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message