commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Neidhart <thomas.neidh...@gmail.com>
Subject Re: [MATH] Jenkins unit test failure
Date Mon, 12 Jan 2015 21:30:26 GMT
On 01/12/2015 10:26 PM, Thomas Neidhart wrote:
> On 01/12/2015 08:09 PM, Phil Steitz wrote:
>> On 1/12/15 11:37 AM, sebb wrote:
>>> On 12 January 2015 at 18:11, Phil Steitz <phil.steitz@gmail.com> wrote:
>>>> On 1/12/15 10:50 AM, sebb wrote:
>>>>> On 11 January 2015 at 22:10, Phil Steitz <phil.steitz@gmail.com>
wrote:
>>>>>> On 1/11/15 11:19 AM, Phil Steitz wrote:
>>>>>>> On 1/10/15 10:49 PM, Phil Steitz wrote:
>>>>>>>> On 1/9/15 6:09 PM, sebb wrote:
>>>>>>>>> On 10 January 2015 at 01:01, Phil Steitz <phil.steitz@gmail.com>
wrote:
>>>>>>>>>> On 1/9/15 5:32 PM, sebb wrote:
>>>>>>>>>>> On 9 January 2015 at 23:48, sebb <sebbaz@gmail.com>
wrote:
>>>>>>>>>>>> Of the last 6 runs, only 1 had a problem
with unit test failures.
>>>>>>>>>>>>
>>>>>>>>>>>> All the builds ran on ubuntu3, apart from
the failure which ran on H10.
>>>>>>>>>>>> This may have some bearing on the result;
I don't yet know.
>>>>>>>>>>>>
>>>>>>>>>>>> I had a quick look at 2 tests that failed:
>>>>>>>>>>>>
>>>>>>>>>>>> SimpleRegressionTest.testPerfect
>>>>>>>>>>>>
>>>>>>>>>>>> SimpleRegressionTest.testPerfectNegative
>>>>>>>>>>>>
>>>>>>>>>>>> Although the test case has some instance
data, these particular tests
>>>>>>>>>>>> do not use any, so it does not look like
a concurrency issue in the
>>>>>>>>>>>> unit test itself.
>>>>>>>>>>>>
>>>>>>>>>>>> The SimpleRegression class has mutable instance
data, but the test
>>>>>>>>>>>> cases create their own instance.
>>>>>>>>>>>>
>>>>>>>>>>>> I don't know anything about the math functions
involved, but it looks
>>>>>>>>>>>> as though Infinity might result from getSignificance()
if
>>>>>>>>>>>> getSlopeStdErr() returns 0, as the latter
is used as a divisor. Or if
>>>>>>>>>>>> the field sumXX is 0 because that is also
used as a divisor.
>>>>>>>>>>>>
>>>>>>>>>>>> Maybe the H10 host has different floating
point hardware?
>>>>>>>>>>>>
>>>>>>>>>>>> I'll try running some more tests on H10.
>>>>>>>>>>> the build failed again on H10; exactly the same
tests failed as before:
>>>>>>>>>>>
>>>>>>>>>>> This test:
>>>>>>>>>>> https://builds.apache.org/job/Commons%20Math%20H10/1/console
>>>>>>>>>>>
>>>>>>>>>>> Previous failure:
>>>>>>>>>>> https://builds.apache.org/job/Commons%20Math/14/console
>>>>>>>>>> This is actually a bug.  Thanks, sebb (and Jenkins)!
>>>>>>>>>>
>>>>>>>>>> Has been here since 1.x.  What is going on is that
the data sets
>>>>>>>>>> used in the test cases are set up to be perfect linear
>>>>>>>>>> relationships, which should in fact lead to mean
square error (and
>>>>>>>>>> hence slope standard error) equal to 0.  The Jenkins
box must be
>>>>>>>>>> getting exact 0.  The funny thing is the test is
there to validate
>>>>>>>>>> correct performance for models like this.  Its success
unfortunately
>>>>>>>>>> depends on poor precision.
>>>>>>>>>>
>>>>>>>>>> I will open a JIRA for this.  I don't think it is
a release blocker
>>>>>>>>>> for 3.4.1, as I am sure you would get the same thing
in any earlier
>>>>>>>>>> version of [math].
>>>>>>>>> OK good to know.
>>>>>>>>>
>>>>>>>>> I'll leave the H10 Jenkins job for now to make it easy
to retest.
>>>>>>>> My first guess here was wrong.  The infinities are being
handled
>>>>>>>> correctly for the JDKs I have.  Something must be going awry
in the
>>>>>>>> t distribution cumulative probability computation for +INF
on the
>>>>>>>> box that is failing.  Is there a way to find out exactly
what JDK
>>>>>>>> and OS version are being used?
>>>>>>> I just committed a test that tests the t distribution computations
>>>>>>> directly.  It seems to have run clean; but the other test ran
clean
>>>>>>> too.  Is there any way to force the build to use the host that
fails?
>>>>>> I can't make any sense of what is going on with the Jenkins builds.
>>>>>> Clean runs and then lots of errors.  This one explains the
>>>>>> SimpleRegression "problem" (which is not a problem with that class
>>>>>> at least)
>>>>>>
>>>>>> testCumulativeProbablilityExtremes(org.apache.commons.math3.distribution.TDistributionTest)
 Time elapsed: 0.001 sec  <<< FAILURE!
>>>>>> java.lang.AssertionError: expected:<1.0> but was:<-Infinity>
>>>>>>         at org.junit.Assert.fail(Assert.java:88)
>>>>>>         at org.junit.Assert.failNotEquals(Assert.java:743)
>>>>>>         at org.junit.Assert.assertEquals(Assert.java:494)
>>>>>>         at org.junit.Assert.assertEquals(Assert.java:592)
>>>>>>         at org.apache.commons.math3.distribution.TDistributionTest.testCumulativeProbablilityExtremes(TDistributionTest.java:109)
>>>>>>
>>>>>> Earlier runs this ran clean. There is nothing non-deterministic about
this test (or quite a few of the others that randomly seem to fail).
>>>>>>
>>>>>> I wonder if we have a bad cpu or something somewhere.
>>>>> AFAICS all the failed builds ran on H10.
>>>>>
>>>>> IMO it is consistent; the apparent randomness comes from the fact the
>>>>> there are several Ubuntu hosts, including H10.
>>>> Am I reading it / looking at the wrong one, or did this one succeed?
>>>>
>>>> https://builds.apache.org/view/All/job/Commons%20Math%20H10/6/
>>>>
>>>> That one was right after I added tests confirming that the t
>>>> distribution cum prob handles INFs correctly.
>>> That did run on H10 and did succeed; I'd not noticed that one before.
>>>
>>> I think it is still true that the failures have only occurred on H10.
>>>
>>> However, the latest one is failing:
>>>
>>> https://builds.apache.org/job/Commons%20Math/24/console
>>>
>>> This is on H11 - I think that's the first time H11 has been used.
>>>
>>> I suppose it's possible that H10 and H11 have a common failing, but it
>>> seems less likely.
>>>
>>> I added a bit more debug - showing the value of sumXX - but that seems
>>> OK on H11.
>>>
>>> I just added a bit more debug.
>>
>> I am pretty sure the SimpleRegressionTest failure is actually cause
>> by the same thing causing the t-distribution test to fail (the
>> reason I added that one).
>>
>> One that is more straightforward to chase is this one, which fails
>> pretty consistently when "bad things happen"
>>
>> testExpInf(org.apache.commons.math3.complex.ComplexTest)  Time elapsed: 0.001 sec
 <<< FAILURE!
>> java.lang.AssertionError: expected:<0.0> but was:<Infinity>
>> 	at org.junit.Assert.fail(Assert.java:88)
>> 	at org.junit.Assert.failNotEquals(Assert.java:743)
>> 	at org.junit.Assert.assertEquals(Assert.java:494)
>> 	at org.junit.Assert.assertEquals(Assert.java:592)
>> 	at org.apache.commons.math3.TestUtils.assertSame(TestUtils.java:76)
>> 	at org.apache.commons.math3.TestUtils.assertSame(TestUtils.java:84)
>> 	at org.apache.commons.math3.complex.ComplexTest.testExpInf(ComplexTest.java:788)
>>
>> I would wager that what is going on here is 0.0 * -INF = INF.
> 
> The output returned by the debug statements added by sebb is:
> 
> expReal=Infinity
> cosImag=0.5403023058681398
> sinImag=0.8414709848078965
> result=(Infinity, Infinity)
> 
> while expReal should be -Infinity.
> 
> of course, Math.exp(Infinity) = Infinity.

oh stupid mistake, please forget my last post.
I messed up expReal with the actual real value.

Thomas

> 
> Thomas
> 
>>
>>
>>>
>>> I can perhaps change the H10 job to additionally run on H11.
>>>
>>>
>>>> Phi
>>>>>> Phil
>>>>>>
>>>>>>
>>>>>>> Phil
>>>>>>>> Phil
>>>>>>>>>> Phil
>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>>>>>
>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>>>>
>>>>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>
>>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>
>>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org


Mime
View raw message