Mailing-List: contact issues-help@commons.apache.org; run by ezmlm
Precedence: bulk
Reply-To: issues@commons.apache.org
Date: Sun, 10 Jan 2016 23:54:39 +0000 (UTC)
From: "Gilles (JIRA)" <jira@apache.org>
To: issues@commons.apache.org
Message-ID: <JIRA.12929009.1452439116000.69454.1452470079867@Atlassian.JIRA>
In-Reply-To: <JIRA.12929009.1452439116000@Atlassian.JIRA>
References: <JIRA.12929009.1452439116000@Atlassian.JIRA>
 <JIRA.12929009.1452439116817@arcas>
Subject: [jira] [Commented] (MATH-1313) Wrong tolerance in some unit tests
 of "RandomGeneratorAbstractTest"
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/MATH-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15091294#comment-15091294 ] 

Gilles commented on MATH-1313:
------------------------------

bq. Testing uniformity seems better than what we currently have

I thought that the current was testing uniformity (unless out of bad luck, the sample follows another distribution that also happens to have a 0.5 mean).
The problem with the current tolerance is that it is wrong: it is proportional to sqrt(N) instead of being proportional to sqrt(1/N).

> Wrong tolerance in some unit tests of "RandomGeneratorAbstractTest"
> -------------------------------------------------------------------
>
>                 Key: MATH-1313
>                 URL: https://issues.apache.org/jira/browse/MATH-1313
>             Project: Commons Math
>          Issue Type: Bug
>            Reporter: Gilles
>            Assignee: Gilles
>            Priority: Minor
>              Labels: unit-test
>             Fix For: 4.0
>
>
> I doubt that the mean check in the unit test below is ever going to trigger an assertion failure...
> {noformat}
>     @Test
>     public void testDoubleDirect() {
>         SummaryStatistics sample = new SummaryStatistics();
>         final int N = 10000;
>         for (int i = 0; i < N; ++i) {
>             sample.addValue(generator.nextDouble());
>         }
>         Assert.assertEquals("Note: This test will fail randomly about 1 in 100 times.",
>                 0.5, sample.getMean(), FastMath.sqrt(N/12.0) * 2.576);
>         Assert.assertEquals(1.0 / (2.0 * FastMath.sqrt(3.0)),
>                      sample.getStandardDeviation(), 0.01);
>     }
> {noformat}
> And similar in "testFloatDirect()".
> I propose the following replacement:
> {noformat}
>     @Test
>     public void testDoubleDirect() {
>         SummaryStatistics sample = new SummaryStatistics();
>         final int N = 100000;
>         for (int i = 0; i < N; ++i) {
>             sample.addValue(generator.nextDouble());
>         }
>         assertUniformInUnitInterval(sample, 0.99);
>     }
> {noformat}
> where "assertUniformInUnitInterval" is defined as:
>    {noformat}
>     /**                                                                                                                                                                                    
>      * Check that the sample follows a uniform distribution on the {@code [0, 1)} interval.                                                                                                
>      *                                                                                                                                                                                     
>      * @param sample Data summary.                                                                                                                                                         
>      * @param confidenceIntervalLevel Confidence level. Must be in {@code (0, 1)} interval.                                                                                                
>      */
>     private void assertUniformInUnitInterval(SummaryStatistics sample,
>                                              double confidenceIntervalLevel) {
>         final int numSamples = (int) sample.getN();
>         final double mean = sample.getMean();
>         final double stddev = sample.getStandardDeviation() / FastMath.sqrt(numSamples);
>         final TDistribution t = new TDistribution(numSamples - 1);
>         final double criticalValue = t.inverseCumulativeProbability(1 - 0.5 * (1 - confidenceIntervalLevel));
>         final double tol = stddev * criticalValue;
>         Assert.assertEquals("mean=" + mean + " tol=" + tol + " (note: This test will fail randomly about " +
>                             (100 * (1 - confidenceIntervalLevel)) + " in 100 times).",
>                             0.5, mean, tol);
>         Assert.assertEquals(FastMath.sqrt(1d / 12), sample.getStandardDeviation(), 0.01);
>     }
> {noformat}
> Please correct if this new test is not what was intended.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)