Mailing-List: contact issues-help@commons.apache.org; run by ezmlm
Precedence: bulk
Reply-To: issues@commons.apache.org
Message-ID: <805976531.420961266773247925.JavaMail.jira@brutus.apache.org>
Date: Sun, 21 Feb 2010 17:27:27 +0000 (UTC)
From: "Niall Pemberton (JIRA)" <jira@apache.org>
To: issues@commons.apache.org
Subject: [jira] Resolved: (LANG-592) RandomUtils tests are failing
 frequently
In-Reply-To: <1367400365.420861266772047904.JavaMail.jira@brutus.apache.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/LANG-592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Niall Pemberton resolved LANG-592.
----------------------------------

    Resolution: Fixed
      Assignee: Niall Pemberton

I have moved the 40+ new chi-square tests into a new test case called RandomUtilsFreqTest and excluded that test from the default Ant/Maven test goals:

    * http://svn.apache.org/viewvc?view=revision&revision=912374

I have added a new Profile to the Maven2 build which just runs the new RandomUtilsFreqTest  test which can be run with the following command:
{code}
    mvn -Ptest-random-freq clean test
{code}

I have added a new Ant target which just runs the new RandomUtilsFreqTest  test which can be run with the following command
{code}
    ant clean test-random-freq
{code}


> RandomUtils tests are failing frequently
> ----------------------------------------
>
>                 Key: LANG-592
>                 URL: https://issues.apache.org/jira/browse/LANG-592
>             Project: Commons Lang
>          Issue Type: Test
>          Components: lang.math.*
>    Affects Versions: 2.x
>            Reporter: Niall Pemberton
>            Assignee: Niall Pemberton
>            Priority: Minor
>             Fix For: 2.5
>
>
> The additionan of 40+ chi-square tests added for RandomUtils have caused the RandomUtilsTest to start failing frequently.
> Phil Steitz investigated this and wrote the following (http://markmail.org/message/mo4qb3qh75nq2kwn) on the mailing list:
> {code}
> The random data tests are failing at a high enough frequency to be
> annoying / alarming to users.
> I investigated the high incidence of test failures and found nothing
> wrong with what the tests are doing and nothing to indicate
> systematic bias in the data being generated; but the addition of 40+
> chi-square tests in the test methods added in r907159 makes the
> probability of failure in a given run > 1/25.  This is why there is
> a high incidence of test failures.
> I verified that failures appear to be evenly distributed (too many,
> too few even/edd, too many, too few above/below range midpoints) and
> that the chisquare statistics are being computed correctly, with the
> right critical values applied.
> If you do cut another RC, I would recommend one of the following:
> 1) Grab / copy and extend [math]'s RetryTestCase (will cut incidence
> of failure in half)
> 2) Disable the stochastic test cases for the release
> 3) Reduce sensitivity of the chi-square test (change to e.g., .0005
> level of significance)
> 4) Reduce the number of tests
> My recommendation is 2) - leave in the source but comment out.  The
> tests are valuable as they would fail regularly and miserably if
> there were systematic bias (as there used to be on odd/even); but
> without reducing significantly the number of tests or the
> sensitivity (or limiting to a single "successful" PRNG sequence),
> there is no way to leave them all in without generating an
> annoyingly high rate of random failures.
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.