harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiao-Feng Li" <xiaofeng...@gmail.com>
Subject Re: [testing] first M7 candidate (r681495) testing status
Date Thu, 21 Aug 2008 06:08:45 GMT
Good finding and interesting!  Thanks, -xiaofeng

On Thu, Aug 21, 2008 at 10:51 AM, chunrong lai <chunronglai@gmail.com> wrote:
> hi,:
>  I got an update about the failure in java.lang.ClassGenericsTest.test_2 in
> DRLVM_test.
>  I see the test case is using some harded-UTF-encoded class-name during the
> test. So the test only passes if the locale is set to "en_US.UTF-8" or
> "zh_CN.utf8". If the locale is set to "C" or "zh_CN.GB2312" it just fails.
> RI also can not pass the test case in such locales. So it looks like a test
> case (or environment setting) problem.
>
>  Thanks.
>
> On 8/12/08, chunrong lai <chunronglai@gmail.com> wrote:
>>
>>
>>
>> Hi, all:
>>
>>   Here is r681495 snapshot testing status:
>> http://people.apache.org/~chunrong/snapshots/r681495/index.html. I am
>> using two platforms (Linux x86, windows x86_64) at the moment. Hopefully we
>> will have other two platforms in future for M8. Well, although we are
>> testing only two platforms for M7, my experience is that the status for
>> another two platforms should be not worse or just include some extra
>> intermittent errors which can be investigated in some later stages.
>>
>>   The following suites passed on Linux x86/Windows x86_64 platforms: Ant
>> Scenario (or self-hosting), Axis application, Dacapo, DRLVM regression
>> tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS VM Test Suite.
>>
>>   Most of the failures are known issues (for M6). Although we can observe
>> 15~20 new issues, those issues happen only in 1 platform and they look more
>> like the intermittent/timeout issues (less reproducible) to me. Overall I
>> myself would like to think r681495 is more stable than M6.
>>
>>   Please add your comments and clarifications (please also see M6 testing
>> results http://people.apache.org/~smishura/r653525/,
>> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand
integrity testing results
>> http://people.apache.org/~chunrong/harmony-integrity/ as a comparison).
>>
>>  1) Classlib:
>>     1.1) Since r644719 (which committed patch for HARMONY-5684)
>>
>>         org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest
>>
>> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
>>
>>          failed in both platforms
>>
>>     1.2) Two failures
>>
>>         org.apache.harmony.luni.tests.java.net.MulticastSocketTest (Failed
>> in windows_x86 running of M6)
>>         tests.api.java.security.PermissionCollectionTest           (Failed
>> in M6)
>>
>>          are observed in Linux x86.
>>
>>  2) DRLVM tests:
>>     2.1) One failure
>>
>>          java.lang.ClassGenericsTest.test_2
>>
>>          is observed in Linux x86 snapshot testing.
>>          I can see some old discussion in the mailing list about that but I
>> am not sure the expected status here.
>>          They should be intemittent errors since the integrity testing just
>> run well mostly.
>>
>>  3) EUTs:
>>
>>     3.1) Linux x86: 99.36%
>>          A recorded JIRA for this suite is HARMONY-2914 which wastes file
>> handlers and makes system unstable.
>>
>>  4) Functional:
>>     4.1) Old regressions on both platforms:
>>          api.java.text.MessageFormat (HARMONY-5430)
>>          api.java.util.jar.Manifest  (HARMONY-5473)
>>          api.java.beans.beancontext.BeanContextTest (also in M6, recorded
>> as  regression caused by changes in locale data)
>>          api.java.beans.persistence.EncoderTest  (also in M6, recorded as
>> regression in the beans module)
>>          api.java.beans.persistence.EncoderDecoderTest (also in M6,
>> regression in the beans module)
>>          reg.vm.btest5625 (also in M6, recorded as intermittent and not
>> reproducible manually)
>>
>>     4.2) Old regressions on 1 platform
>>          api.java.rmi.basicexception (ERROR in Linux x86, HARMONY-5823)
>>          api.java.rmi.basicregistry.RemoteServerTest (ERROR in Linux x86,
>> HARMONY-5823)
>>          jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows x86_64, in M6
>> it is recorded as regression since M4)
>>          java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR on Linux x86,
>> recorded as Timeout, not reproducible
>>  in M6)
>>          reg.vm.btest5717 (ERROR on Windows X86_64, recorded as "timeout,
>> the test is too heavy" in M6)
>>          jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1 (FAILED in
>> windows x86_64, recorded as "looks like
>> issue in test" in M6)
>>          jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on windows x86_64,
>> in M6 it is recorded as not regression and started to fail on M5)
>>          reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64, recorded also
>> failed on M3 and might be issue with the test)
>>
>>     4.3) New regressions on 1 platform (looks intermittent)
>>          reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
>>          func.reg.jit.btest5710.Btest5710 (FAILED in Linux x86)
>>          api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06 (ERROR
>> in Linux x86)
>>          api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05 (ERROR
>> in Linux x86)
>>
>>  5) JDKTools Tests:
>>     Several timeouts are observed in Linux x86 snapshot running. They are:
>>
>> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001
>>
>> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002
>>
>> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003
>>
>> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004
>>
>> org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04
>>
>> org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001
>>
>> org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001
>>
>> org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume
>>
>> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001
>>
>> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
>>     The Linux-only timeouts are also observed in the integrity testing
>> results.
>>     JIRA HARMONY-5833 has been filed for one of them.
>>
>>  6) JettyScenario:
>>     The Linux x86 running failed because of the unresolved HARMONY-5219.
>>
>>  7) Reliability:
>>     Several failures are observed in windows x86_64 running.
>>     7.1) Old regressions
>>          api.net.DatagramTest (HARMONY-5531)
>>          api.text.DecimalFormat_Locales - (in M6 it is recorded as also
>> intermittent in M5)
>>
>>     7.2) New/intemittent regressions
>>          api.kernel.thread.ThreadLocalTest.ThreadLocalTest
>>          api.kernel.exec.RunExec
>>
>>  8) Stress
>>     Different test cases failed on different platforms.
>>     8.1) Timeouts on Linux x86.
>>
>> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003
>>
>> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007
>>
>> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009
>>
>> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011
>>
>>     8.2) Failed cases on Windows x86_64 with unknown reasons.
>>
>> stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2
>>
>> stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses
>>
>>     I have not found records for them.
>>
>>  9) Strut_test
>>     Broken with same error report as M6.
>>
>>  10) Eclipse Hello World Application.
>>     Although the testing framework just reports EUT-API status as "PASSED".
>> A fresh workspace running just failed in configuration stage (
>> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e)
>> since SVN commit 641928 (which committed patch for HARMONY-4569).
>>
>> thanks,
>> chunrong
>>
>



-- 
http://xiao-feng.blogspot.com

Mime
View raw message