commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Phil Steitz <phil.ste...@gmail.com>
Subject Re: [MATH] Jenkins unit test failure
Date Mon, 12 Jan 2015 19:09:50 GMT
On 1/12/15 11:37 AM, sebb wrote:
> On 12 January 2015 at 18:11, Phil Steitz <phil.steitz@gmail.com> wrote:
>> On 1/12/15 10:50 AM, sebb wrote:
>>> On 11 January 2015 at 22:10, Phil Steitz <phil.steitz@gmail.com> wrote:
>>>> On 1/11/15 11:19 AM, Phil Steitz wrote:
>>>>> On 1/10/15 10:49 PM, Phil Steitz wrote:
>>>>>> On 1/9/15 6:09 PM, sebb wrote:
>>>>>>> On 10 January 2015 at 01:01, Phil Steitz <phil.steitz@gmail.com>
wrote:
>>>>>>>> On 1/9/15 5:32 PM, sebb wrote:
>>>>>>>>> On 9 January 2015 at 23:48, sebb <sebbaz@gmail.com>
wrote:
>>>>>>>>>> Of the last 6 runs, only 1 had a problem with unit
test failures.
>>>>>>>>>>
>>>>>>>>>> All the builds ran on ubuntu3, apart from the failure
which ran on H10.
>>>>>>>>>> This may have some bearing on the result; I don't
yet know.
>>>>>>>>>>
>>>>>>>>>> I had a quick look at 2 tests that failed:
>>>>>>>>>>
>>>>>>>>>> SimpleRegressionTest.testPerfect
>>>>>>>>>>
>>>>>>>>>> SimpleRegressionTest.testPerfectNegative
>>>>>>>>>>
>>>>>>>>>> Although the test case has some instance data, these
particular tests
>>>>>>>>>> do not use any, so it does not look like a concurrency
issue in the
>>>>>>>>>> unit test itself.
>>>>>>>>>>
>>>>>>>>>> The SimpleRegression class has mutable instance data,
but the test
>>>>>>>>>> cases create their own instance.
>>>>>>>>>>
>>>>>>>>>> I don't know anything about the math functions involved,
but it looks
>>>>>>>>>> as though Infinity might result from getSignificance()
if
>>>>>>>>>> getSlopeStdErr() returns 0, as the latter is used
as a divisor. Or if
>>>>>>>>>> the field sumXX is 0 because that is also used as
a divisor.
>>>>>>>>>>
>>>>>>>>>> Maybe the H10 host has different floating point hardware?
>>>>>>>>>>
>>>>>>>>>> I'll try running some more tests on H10.
>>>>>>>>> the build failed again on H10; exactly the same tests
failed as before:
>>>>>>>>>
>>>>>>>>> This test:
>>>>>>>>> https://builds.apache.org/job/Commons%20Math%20H10/1/console
>>>>>>>>>
>>>>>>>>> Previous failure:
>>>>>>>>> https://builds.apache.org/job/Commons%20Math/14/console
>>>>>>>> This is actually a bug.  Thanks, sebb (and Jenkins)!
>>>>>>>>
>>>>>>>> Has been here since 1.x.  What is going on is that the data
sets
>>>>>>>> used in the test cases are set up to be perfect linear
>>>>>>>> relationships, which should in fact lead to mean square error
(and
>>>>>>>> hence slope standard error) equal to 0.  The Jenkins box
must be
>>>>>>>> getting exact 0.  The funny thing is the test is there to
validate
>>>>>>>> correct performance for models like this.  Its success unfortunately
>>>>>>>> depends on poor precision.
>>>>>>>>
>>>>>>>> I will open a JIRA for this.  I don't think it is a release
blocker
>>>>>>>> for 3.4.1, as I am sure you would get the same thing in any
earlier
>>>>>>>> version of [math].
>>>>>>> OK good to know.
>>>>>>>
>>>>>>> I'll leave the H10 Jenkins job for now to make it easy to retest.
>>>>>> My first guess here was wrong.  The infinities are being handled
>>>>>> correctly for the JDKs I have.  Something must be going awry in the
>>>>>> t distribution cumulative probability computation for +INF on the
>>>>>> box that is failing.  Is there a way to find out exactly what JDK
>>>>>> and OS version are being used?
>>>>> I just committed a test that tests the t distribution computations
>>>>> directly.  It seems to have run clean; but the other test ran clean
>>>>> too.  Is there any way to force the build to use the host that fails?
>>>> I can't make any sense of what is going on with the Jenkins builds.
>>>> Clean runs and then lots of errors.  This one explains the
>>>> SimpleRegression "problem" (which is not a problem with that class
>>>> at least)
>>>>
>>>> testCumulativeProbablilityExtremes(org.apache.commons.math3.distribution.TDistributionTest)
 Time elapsed: 0.001 sec  <<< FAILURE!
>>>> java.lang.AssertionError: expected:<1.0> but was:<-Infinity>
>>>>         at org.junit.Assert.fail(Assert.java:88)
>>>>         at org.junit.Assert.failNotEquals(Assert.java:743)
>>>>         at org.junit.Assert.assertEquals(Assert.java:494)
>>>>         at org.junit.Assert.assertEquals(Assert.java:592)
>>>>         at org.apache.commons.math3.distribution.TDistributionTest.testCumulativeProbablilityExtremes(TDistributionTest.java:109)
>>>>
>>>> Earlier runs this ran clean. There is nothing non-deterministic about this
test (or quite a few of the others that randomly seem to fail).
>>>>
>>>> I wonder if we have a bad cpu or something somewhere.
>>> AFAICS all the failed builds ran on H10.
>>>
>>> IMO it is consistent; the apparent randomness comes from the fact the
>>> there are several Ubuntu hosts, including H10.
>> Am I reading it / looking at the wrong one, or did this one succeed?
>>
>> https://builds.apache.org/view/All/job/Commons%20Math%20H10/6/
>>
>> That one was right after I added tests confirming that the t
>> distribution cum prob handles INFs correctly.
> That did run on H10 and did succeed; I'd not noticed that one before.
>
> I think it is still true that the failures have only occurred on H10.
>
> However, the latest one is failing:
>
> https://builds.apache.org/job/Commons%20Math/24/console
>
> This is on H11 - I think that's the first time H11 has been used.
>
> I suppose it's possible that H10 and H11 have a common failing, but it
> seems less likely.
>
> I added a bit more debug - showing the value of sumXX - but that seems
> OK on H11.
>
> I just added a bit more debug.

I am pretty sure the SimpleRegressionTest failure is actually cause
by the same thing causing the t-distribution test to fail (the
reason I added that one).

One that is more straightforward to chase is this one, which fails
pretty consistently when "bad things happen"

testExpInf(org.apache.commons.math3.complex.ComplexTest)  Time elapsed: 0.001 sec  <<<
FAILURE!
java.lang.AssertionError: expected:<0.0> but was:<Infinity>
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.failNotEquals(Assert.java:743)
	at org.junit.Assert.assertEquals(Assert.java:494)
	at org.junit.Assert.assertEquals(Assert.java:592)
	at org.apache.commons.math3.TestUtils.assertSame(TestUtils.java:76)
	at org.apache.commons.math3.TestUtils.assertSame(TestUtils.java:84)
	at org.apache.commons.math3.complex.ComplexTest.testExpInf(ComplexTest.java:788)

I would wager that what is going on here is 0.0 * -INF = INF.


>
> I can perhaps change the H10 job to additionally run on H11.
>
>
>> Phi
>>>> Phil
>>>>
>>>>
>>>>> Phil
>>>>>> Phil
>>>>>>>> Phil
>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>>
>>>>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org


Mime
View raw message