harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oliver Deakin <oliver.dea...@googlemail.com>
Subject Re: [testing] first M7 candidate (r681495) testing status
Date Wed, 13 Aug 2008 10:54:57 GMT
I agree - this can probably be left until after the M7 code freeze.
As Sian says, the DRLVM Throwable implementation of initCause() can 
probably just be taken from the luni-kernel version in classlib, as this 
appears to match the spec.

Regards,
Oliver

Sian January wrote:
> Oliver has helped me track this down this morning, and it's occuring
> because there are differences between the versions of
> Throwable.initCause(..) in luni and in DRLVM.  According to the spec,
> I think the test should be changed to check for IllegalStateException
> instead of IllegalArgumentException and then I think the version of
> Throwable in DRLVM should be changed to behave the same as the one in
> luni.  However I don't think this is a big deal for M7 because I don't
> think it's very serious and it's not a regression since M6.  What does
> anyone else think?
>
> Sian
>
> On 13/08/2008, Sian January <sianjanuary@googlemail.com> wrote:
>   
>> I think the code is right and the test is wrong here, because the
>> Javadoc for ServerCloneException says that "Invoking the method
>> Throwable.initCause(Throwable) on an instance of ServerCloneException
>> always throws IllegalStateException."
>>
>> What is really confusing me is how this can be passing for anyone
>> else... The exclude list hasn't changed recently and I don't see how
>> it could be VM dependent either.
>>
>>
>> On 13/08/2008, Nathan Beyer <ndbeyer@apache.org> wrote:
>>     
>>> On Tue, Aug 12, 2008 at 10:18 AM, Sian January
>>> <sianjanuary@googlemail.com>wrote:
>>>
>>>       
>>>> I am not seeing the management failures on my local (Windows) machine,
>>>> but I'm seeing an rmi failure:
>>>>
>>>>
>>>> org.apache.harmony.rmi.server.ServerCloneExceptionTest.test_Constructor_String
>>>> Cause already initialized
>>>> java.lang.IllegalStateException: Cause already initialized
>>>> at java.lang.Throwable.initCause(Throwable.java:293)
>>>> at
>>>> org.apache.harmony.rmi.server.ServerCloneExceptionTest.test_Constructor_String(ServerCloneExceptionTest.java:70)
>>>> at java.lang.reflect.AccessibleObject.invokeV(AccessibleObject.java:25)
>>>>         
>>> This test [1] is creating an exception and then passing itself into
>>> 'initCause', which should throw IllegalArgumentException. Looking at the
>>> code though, it does look like there's a bug, as 'null' is being passed to
>>> the super() constructor, which will mess up Throwable, since it initializes
>>> 'cause' to 'this', so Throwable will think the cause is already set.
>>>
>>> Did this test recently get removed from an exclusion list?
>>>
>>>
>>> [1]
>>> http://svn.apache.org/viewvc/harmony/enhanced/classlib/trunk/modules/rmi/src/test/api/java/org/apache/harmony/rmi/server/ServerCloneExceptionTest.java?view=annotate
>>>
>>>       
>>>> and an intermittent failure in java.net:
>>>>
>>>>
>>>> org.apache.harmony.luni.tests.java.net.URLConnectionTest.test_getInputStream
>>>>
>>>> Error receiving content coded data:
>>>> junit.framework.AssertionFailedError: Error receiving content coded data:
>>>> at tests.support.Support_HttpTests.runTests(Support_HttpTests.java:15)
>>>> at
>>>> org.apache.harmony.luni.tests.java.net.URLConnectionTest.test_getInputStream(URLConnectionTest.java:632)
>>>> at java.lang.reflect.AccessibleObject.invokeV(AccessibleObject.java:25)
>>>>
>>>> Is anyone else seeing these?
>>>>
>>>> Sian
>>>>
>>>> On 12/08/2008, chunrong lai <chunronglai@gmail.com> wrote:
>>>>         
>>>>>   Thanks.
>>>>>   I think the standard scripts produce that site although I always added
>>>>> publish/e-mail notification setup before install/run the test suites.
>>>>>   It is understandable that some tests in classlib-test and drlvm-test
>>>>>           
>>>> get
>>>>         
>>>>> failing report. We just need to understand the failures.
>>>>>   As reported in M6,
>>>>> org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest,
>>>>>
>>>>>           
>>>> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
>>>>         
>>>>> etc fail classlib-test.
>>>>>   Also we observed that java.lang.ClassGenericsTest.test_2 sometimes
fail
>>>>> drlvm-test.
>>>>>   Another timeout issue of drlvm-test comes from
>>>>> thread.SmallStackThreadTest_jit, as discussed in HARMONY-4601, the test
>>>>>           
>>>> case
>>>>         
>>>>> may run very slow when system resource is tight and finally introduce
a
>>>>> timeout. Re-running the suite generally fixes the timeout.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 8/12/08, Nathan Beyer <ndbeyer@apache.org> wrote:
>>>>>           
>>>>>> It's the tests that are failing - both classlib-test and drlvm-test.
>>>>>>
>>>>>> What's the specific integrity setup that's being used for that site?
>>>>>>             
>>>> Does
>>>>         
>>>>>> the standard out-of-the-box buildtest scripts produce that site?
>>>>>>
>>>>>> -Nathan
>>>>>>
>>>>>> On Mon, Aug 11, 2008 at 9:05 PM, chunrong lai <chunronglai@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>             
>>>>>>> hi, Nathan:
>>>>>>>     Steps to run testing cycle are listed in
>>>>>>> http://harmony.apache.org/subcomponents/buildtest/index.html.
One
>>>>>>> integrity
>>>>>>> testing example is
>>>>>>>
>>>>>>>               
>>>> http://harmony.apache.org/subcomponents/buildtest/howto.html#Extended.
>>>>         
>>>>>>>      What is the error message you met (in what step)? What is
the
>>>>>>>               
>>>>>> content
>>>>>>             
>>>>>>> of the framework.local.properties file? I think that will help
to
>>>>>>>               
>>>> figure
>>>>         
>>>>>>> out
>>>>>>> the problem.
>>>>>>>
>>>>>>>
>>>>>>> On 8/12/08, Nathan Beyer <ndbeyer@apache.org> wrote:
>>>>>>>               
>>>>>>>> What's being used to produce the integrity tests? I have
an x86_64
>>>>>>>>                 
>>>>>> Linux
>>>>>>             
>>>>>>>> box
>>>>>>>> that I can dedicate to testing. I've tried just using buildtest
>>>>>>>>                 
>>>> with
>>>>         
>>>>>>>> 'classlib,drlvm,classib-test,drlvm-test', but I haven't gotten
a
>>>>>>>>                 
>>>> clean
>>>>         
>>>>>>> pass
>>>>>>>               
>>>>>>>> yet.
>>>>>>>>
>>>>>>>> -Nathan
>>>>>>>>
>>>>>>>> On Mon, Aug 11, 2008 at 12:26 PM, chunrong lai <
>>>>>>>>                 
>>>> chunronglai@gmail.com
>>>>         
>>>>>>>>> wrote:
>>>>>>>>>                   
>>>>>>>>> Hi, all:
>>>>>>>>>
>>>>>>>>>  Here is r681495 snapshot testing status:
>>>>>>>>> http://people.apache.org/~chunrong/snapshots/r681495/index.html<http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>
>>>>>>>>>                   
>>>> <
>>>>         
>>>>>> http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>
>>>>>>             
>>>>>>> <
>>>>>>>               
>>>>>>>> http://people.apache.org/%7Echunrong/snapshots/r681495/index.html
>>>>>>>>                 
>>>>> .
>>>>>           
>>>>>>>>> I am using
>>>>>>>>> two platforms (Linux x86, windows x86_64) at the moment.
>>>>>>>>>                   
>>>> Hopefully we
>>>>         
>>>>>>>> will
>>>>>>>>                 
>>>>>>>>> have other two platforms in future for M8. Well, although
we are
>>>>>>>>>                   
>>>>>>> testing
>>>>>>>               
>>>>>>>>> only two platforms for M7, my experience is that the
status for
>>>>>>>>>                   
>>>>>> another
>>>>>>             
>>>>>>>> two
>>>>>>>>                 
>>>>>>>>> platforms should be not worse or just include some extra
>>>>>>>>>                   
>>>> intermittent
>>>>         
>>>>>>>>> errors
>>>>>>>>> which can be investigated in some later stages.
>>>>>>>>>
>>>>>>>>>  The following suites passed on Linux x86/Windows x86_64
>>>>>>>>>                   
>>>> platforms:
>>>>         
>>>>>> Ant
>>>>>>             
>>>>>>>>> Scenario (or self-hosting), Axis application, Dacapo,
DRLVM
>>>>>>>>>                   
>>>>>> regression
>>>>>>             
>>>>>>>>> tests, Geronimo Unit Tests, Scimark, Tomcat scenario,
VTS VM Test
>>>>>>>>>                   
>>>>>>> Suite.
>>>>>>>               
>>>>>>>>>  Most of the failures are known issues (for M6). Although
we can
>>>>>>>>>                   
>>>>>>> observe
>>>>>>>               
>>>>>>>>> 15~20 new issues, those issues happen only in 1 platform
and they
>>>>>>>>>                   
>>>>>> look
>>>>>>             
>>>>>>>> more
>>>>>>>>                 
>>>>>>>>> like the intermittent/timeout issues (less reproducible)
to me.
>>>>>>>>>                   
>>>>>> Overall
>>>>>>             
>>>>>>> I
>>>>>>>               
>>>>>>>>> myself would like to think r681495 is more stable than
M6.
>>>>>>>>>
>>>>>>>>>  Please add your comments and clarifications (please
also see M6
>>>>>>>>>                   
>>>>>>> testing
>>>>>>>               
>>>>>>>>> results http://people.apache.org/~smishura/r653525/<http://people.apache.org/%7Esmishura/r653525/>
>>>>>>>>>                   
>>>> <
>>>>         
>>>>>> http://people.apache.org/%7Esmishura/r653525/>
>>>>>>             
>>>>>>> <
>>>>>>>               
>>>>>>>> http://people.apache.org/%7Esmishura/r653525/>
>>>>>>>>                 
>>>>>>>>> ,
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand
>>>>         
>>>>>>>>> integrity testing results
>>>>>>>>> http://people.apache.org/~chunrong/harmony-integrity/<http://people.apache.org/%7Echunrong/harmony-integrity/>
>>>>>>>>>                   
>>>> <
>>>>         
>>>>>> http://people.apache.org/%7Echunrong/harmony-integrity/>
>>>>>>             
>>>>>>> <
>>>>>>>               
>>>>>>>> http://people.apache.org/%7Echunrong/harmony-integrity/>as
a
>>>>>>>>                 
>>>>>>> comparison).
>>>>>>>               
>>>>>>>>>  1) Classlib:
>>>>>>>>>    1.1) Since r644719 (which committed patch for HARMONY-5684)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>  org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest
>>>>         
>>>>>>>>>
>>>>>>>>>                   
>>>> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
>>>>         
>>>>>>>>>         failed in both platforms
>>>>>>>>>
>>>>>>>>>    1.2) Two failures
>>>>>>>>>
>>>>>>>>>        org.apache.harmony.luni.tests.java.net.MulticastSocketTest
>>>>>>>>>                   
>>>>>>> (Failed
>>>>>>>               
>>>>>>>>> in windows_x86 running of M6)
>>>>>>>>>        tests.api.java.security.PermissionCollectionTest
>>>>>>>>>                   
>>>>>>> (Failed
>>>>>>>               
>>>>>>>>> in M6)
>>>>>>>>>
>>>>>>>>>         are observed in Linux x86.
>>>>>>>>>
>>>>>>>>>  2) DRLVM tests:
>>>>>>>>>    2.1) One failure
>>>>>>>>>
>>>>>>>>>         java.lang.ClassGenericsTest.test_2
>>>>>>>>>
>>>>>>>>>         is observed in Linux x86 snapshot testing.
>>>>>>>>>         I can see some old discussion in the mailing
list about
>>>>>>>>>                   
>>>> that
>>>>         
>>>>>>> but
>>>>>>>               
>>>>>>>> I
>>>>>>>>                 
>>>>>>>>> am not sure the expected status here.
>>>>>>>>>         They should be intemittent errors since the integrity
>>>>>>>>>                   
>>>> testing
>>>>         
>>>>>>>> just
>>>>>>>>                 
>>>>>>>>> run well mostly.
>>>>>>>>>
>>>>>>>>>  3) EUTs:
>>>>>>>>>
>>>>>>>>>    3.1) Linux x86: 99.36%
>>>>>>>>>         A recorded JIRA for this suite is HARMONY-2914
which
>>>>>>>>>                   
>>>> wastes
>>>>         
>>>>>>> file
>>>>>>>               
>>>>>>>>> handlers and makes system unstable.
>>>>>>>>>
>>>>>>>>>  4) Functional:
>>>>>>>>>    4.1) Old regressions on both platforms:
>>>>>>>>>         api.java.text.MessageFormat (HARMONY-5430)
>>>>>>>>>         api.java.util.jar.Manifest  (HARMONY-5473)
>>>>>>>>>         api.java.beans.beancontext.BeanContextTest (also
in M6,
>>>>>>>>>                   
>>>>>>> recorded
>>>>>>>               
>>>>>>>>> as  regression caused by changes in locale data)
>>>>>>>>>         api.java.beans.persistence.EncoderTest  (also
in M6,
>>>>>>>>>                   
>>>> recorded
>>>>         
>>>>>>> as
>>>>>>>               
>>>>>>>>> regression in the beans module)
>>>>>>>>>         api.java.beans.persistence.EncoderDecoderTest
(also in
>>>>>>>>>                   
>>>> M6,
>>>>         
>>>>>>>>> regression in the beans module)
>>>>>>>>>         reg.vm.btest5625 (also in M6, recorded as intermittent
>>>>>>>>>                   
>>>> and
>>>>         
>>>>>> not
>>>>>>             
>>>>>>>>> reproducible manually)
>>>>>>>>>
>>>>>>>>>    4.2) Old regressions on 1 platform
>>>>>>>>>         api.java.rmi.basicexception (ERROR in Linux x86,
>>>>>>>>>                   
>>>>>> HARMONY-5823)
>>>>>>             
>>>>>>>>>         api.java.rmi.basicregistry.RemoteServerTest (ERROR
in
>>>>>>>>>                   
>>>> Linux
>>>>         
>>>>>>> x86,
>>>>>>>               
>>>>>>>>> HARMONY-5823)
>>>>>>>>>         jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows
>>>>>>>>>                   
>>>> x86_64, in
>>>>         
>>>>>>> M6
>>>>>>>               
>>>>>>>> it
>>>>>>>>                 
>>>>>>>>> is recorded as regression since M4)
>>>>>>>>>         java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR
on
>>>>>>>>>                   
>>>> Linux
>>>>         
>>>>>>> x86,
>>>>>>>               
>>>>>>>>> recorded as Timeout, not reproducible
>>>>>>>>>  in M6)
>>>>>>>>>         reg.vm.btest5717 (ERROR on Windows X86_64, recorded
as
>>>>>>>>>                   
>>>>>>> "timeout,
>>>>>>>               
>>>>>>>>> the test is too heavy" in M6)
>>>>>>>>>         jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1
>>>>>>>>>                   
>>>> (FAILED
>>>>         
>>>>>>> in
>>>>>>>               
>>>>>>>>> windows x86_64, recorded as "looks like
>>>>>>>>> issue in test" in M6)
>>>>>>>>>         jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED
on windows
>>>>>>>>>                   
>>>>>>> x86_64,
>>>>>>>               
>>>>>>>> in
>>>>>>>>                 
>>>>>>>>> M6 it is recorded as not regression and started to fail
on M5)
>>>>>>>>>         reg.vm.btest6353.Btest6353 (ERROR on Windows
x86_64,
>>>>>>>>>                   
>>>> recorded
>>>>         
>>>>>>>> also
>>>>>>>>                 
>>>>>>>>> failed on M3 and might be issue with the test)
>>>>>>>>>
>>>>>>>>>    4.3) New regressions on 1 platform (looks intermittent)
>>>>>>>>>         reg.jit.btest8029.Btest8029 (FAILED in Linux
x86)
>>>>>>>>>         func.reg.jit.btest5710.Btest5710 (FAILED in Linux
x86)
>>>>>>>>>
>>>>>>>>>                   
>>>> api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06
>>>>         
>>>>>>> (ERROR
>>>>>>>               
>>>>>>>>> in Linux x86)
>>>>>>>>>
>>>>>>>>>                   
>>>> api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05
>>>>         
>>>>>>> (ERROR
>>>>>>>               
>>>>>>>>> in Linux x86)
>>>>>>>>>
>>>>>>>>>  5) JDKTools Tests:
>>>>>>>>>    Several timeouts are observed in Linux x86 snapshot
running.
>>>>>>>>>                   
>>>> They
>>>>         
>>>>>>> are:
>>>>>>>               
>>>>>>>>>
>>>>>>>>>                   
>>>> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001
>>>>         
>>>>>>>>>
>>>>>>>>>                   
>>>> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002
>>>>         
>>>>>>>>>
>>>>>>>>>                   
>>>> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003
>>>>         
>>>>>>>>>
>>>>>>>>>                   
>>>> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004
>>>>         
>>>>>>>>>
>>>>>>>>>                   
>>>> org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04
>>>>         
>>>>>>>>>
>>>>>>>>>                   
>>>> org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001
>>>>         
>>>>>>>>>
>>>>>>>>>                   
>>>> org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001
>>>>         
>>>>>>>>>                   
>>>> org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume
>>>>         
>>>>>>>>>
>>>>>>>>>                   
>>>> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001
>>>>         
>>>>>>>>>
>>>>>>>>>                   
>>>> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
>>>>         
>>>>>>>>>    The Linux-only timeouts are also observed in the integrity
>>>>>>>>>                   
>>>> testing
>>>>         
>>>>>>>>> results.
>>>>>>>>>    JIRA HARMONY-5833 has been filed for one of them.
>>>>>>>>>
>>>>>>>>>  6) JettyScenario:
>>>>>>>>>    The Linux x86 running failed because of the unresolved
>>>>>>>>>                   
>>>>>> HARMONY-5219.
>>>>>>             
>>>>>>>>>  7) Reliability:
>>>>>>>>>    Several failures are observed in windows x86_64 running.
>>>>>>>>>    7.1) Old regressions
>>>>>>>>>         api.net.DatagramTest (HARMONY-5531)
>>>>>>>>>         api.text.DecimalFormat_Locales - (in M6 it is
recorded as
>>>>>>>>>                   
>>>>>> also
>>>>>>             
>>>>>>>>> intermittent in M5)
>>>>>>>>>
>>>>>>>>>    7.2) New/intemittent regressions
>>>>>>>>>         api.kernel.thread.ThreadLocalTest.ThreadLocalTest
>>>>>>>>>         api.kernel.exec.RunExec
>>>>>>>>>
>>>>>>>>>  8) Stress
>>>>>>>>>    Different test cases failed on different platforms.
>>>>>>>>>    8.1) Timeouts on Linux x86.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003
>>>>         
>>>>>>>>>
>>>>>>>>>                   
>>>> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007
>>>>         
>>>>>>>>>
>>>>>>>>>                   
>>>> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009
>>>>         
>>>>>>>>>
>>>>>>>>>                   
>>>> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011
>>>>         
>>>>>>>>>    8.2) Failed cases on Windows x86_64 with unknown reasons.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>> stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2
>>>>         
>>>>>>>>>
>>>>>>>>>                   
>>>> stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses
>>>>         
>>>>>>>>>    I have not found records for them.
>>>>>>>>>
>>>>>>>>>  9) Strut_test
>>>>>>>>>    Broken with same error report as M6.
>>>>>>>>>
>>>>>>>>>  10) Eclipse Hello World Application.
>>>>>>>>>    Although the testing framework just reports EUT-API
status as
>>>>>>>>>                   
>>>>>>>> "PASSED".
>>>>>>>>                 
>>>>>>>>> A fresh workspace running just failed in configuration
stage (
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e
>>>>         
>>>>>>>>> )
>>>>>>>>> since SVN commit 641928 (which committed patch for HARMONY-4569).
>>>>>>>>>
>>>>>>>>> thanks,
>>>>>>>>> chunrong
>>>>>>>>>
>>>>>>>>>                   
>>>>
>>>> --
>>>> Unless stated otherwise above:
>>>> IBM United Kingdom Limited - Registered in England and Wales with number
>>>> 741598.
>>>> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>>>>
>>>>         
>> --
>> Unless stated otherwise above:
>> IBM United Kingdom Limited - Registered in England and Wales with number 741598.
>> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>>
>>     
>
>
>   

-- 
Oliver Deakin
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


Mime
View raw message