commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gilles Sadowski <>
Subject Re: Bug in "RandomDataTest" ?
Date Sun, 29 Jan 2012 23:40:26 GMT

> >> My worry is that the test fails for many choices of seeds. And I don't know
> >> how to figure out whether the unit test is wrong, or the code is wrong, or
> >> it is normal that the choice of seed has such an influence on the test (in
> >> which case it should be documented that not all seeds are equal...).
> >>
> >>
> > I've had the same concerns while working on MATH-692. I do not have a
> > satisfactory answer, though. People seemed happy to just use a seed
> > that works, but I'm not sure we are proving anything at all. Maybe
> > what should be done is compute a probability of failure of the test by
> > sampling the seeds. Of course, the unit tests would become quite
> I don't think it will prove anything more. We seem to lack the skills to
> understand these failures. Perhaps we could ask ourselves for help on
> the users list ? We could ask for someone knowledgeable to check our
> code, our tests, and the tests that fail in order to identify the problem.
> > expensive, so we might think of a way to carry them out optionnally.
> > S├ębastien
> > 

Instinctively (although maybe naively) I agree with S├ębastien; I was also
thinking of doing the test he described (but, for sure, it won't be unit
What is the test supposed to be demonstrating? I thought that the answer to
that question was: that there is a very high probablility that the
distribution of the random numbers generated is indeed uniform.
So, by performing many runs (e.g. selecting, say, "x" consecutive seeds and
requesting "y" numbers from the RNG), we can know how many of the seeds do
not produce a uniform distribution (by counting the number of failures
reported by the "assertChiSquareAccept"), and the question is whether this
value is expected for the RNG at hand.


P.S. I note that the description of RNGs refers to cycles (which might be
     more or less long) but the failures we see do not seem related to this
     kind of non-random behaviour since the number of samples is small.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message