harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "chunrong lai" <chunrong...@gmail.com>
Subject [testing] first M7 candidate (r681495) testing status
Date Mon, 11 Aug 2008 17:26:21 GMT
Hi, all:

  Here is r681495 snapshot testing status:
http://people.apache.org/~chunrong/snapshots/r681495/index.html. I am using
two platforms (Linux x86, windows x86_64) at the moment. Hopefully we will
have other two platforms in future for M8. Well, although we are testing
only two platforms for M7, my experience is that the status for another two
platforms should be not worse or just include some extra intermittent errors
which can be investigated in some later stages.

  The following suites passed on Linux x86/Windows x86_64 platforms: Ant
Scenario (or self-hosting), Axis application, Dacapo, DRLVM regression
tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS VM Test Suite.

  Most of the failures are known issues (for M6). Although we can observe
15~20 new issues, those issues happen only in 1 platform and they look more
like the intermittent/timeout issues (less reproducible) to me. Overall I
myself would like to think r681495 is more stable than M6.

  Please add your comments and clarifications (please also see M6 testing
results http://people.apache.org/~smishura/r653525/,
http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand
integrity testing results
http://people.apache.org/~chunrong/harmony-integrity/ as a comparison).

 1) Classlib:
    1.1) Since r644719 (which committed patch for HARMONY-5684)

        org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest

org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest

         failed in both platforms

    1.2) Two failures

        org.apache.harmony.luni.tests.java.net.MulticastSocketTest (Failed
in windows_x86 running of M6)
        tests.api.java.security.PermissionCollectionTest           (Failed
in M6)

         are observed in Linux x86.

 2) DRLVM tests:
    2.1) One failure

         java.lang.ClassGenericsTest.test_2

         is observed in Linux x86 snapshot testing.
         I can see some old discussion in the mailing list about that but I
am not sure the expected status here.
         They should be intemittent errors since the integrity testing just
run well mostly.

 3) EUTs:

    3.1) Linux x86: 99.36%
         A recorded JIRA for this suite is HARMONY-2914 which wastes file
handlers and makes system unstable.

 4) Functional:
    4.1) Old regressions on both platforms:
         api.java.text.MessageFormat (HARMONY-5430)
         api.java.util.jar.Manifest  (HARMONY-5473)
         api.java.beans.beancontext.BeanContextTest (also in M6, recorded
as  regression caused by changes in locale data)
         api.java.beans.persistence.EncoderTest  (also in M6, recorded as
regression in the beans module)
         api.java.beans.persistence.EncoderDecoderTest (also in M6,
regression in the beans module)
         reg.vm.btest5625 (also in M6, recorded as intermittent and not
reproducible manually)

    4.2) Old regressions on 1 platform
         api.java.rmi.basicexception (ERROR in Linux x86, HARMONY-5823)
         api.java.rmi.basicregistry.RemoteServerTest (ERROR in Linux x86,
HARMONY-5823)
         jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows x86_64, in M6 it
is recorded as regression since M4)
         java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR on Linux x86,
recorded as Timeout, not reproducible
 in M6)
         reg.vm.btest5717 (ERROR on Windows X86_64, recorded as "timeout,
the test is too heavy" in M6)
         jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1 (FAILED in
windows x86_64, recorded as "looks like
issue in test" in M6)
         jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on windows x86_64, in
M6 it is recorded as not regression and started to fail on M5)
         reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64, recorded also
failed on M3 and might be issue with the test)

    4.3) New regressions on 1 platform (looks intermittent)
         reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
         func.reg.jit.btest5710.Btest5710 (FAILED in Linux x86)
         api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06 (ERROR
in Linux x86)
         api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05 (ERROR
in Linux x86)

 5) JDKTools Tests:
    Several timeouts are observed in Linux x86 snapshot running. They are:

org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001

org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002

org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003

org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004

org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04

org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001

org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001

org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume

org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001

org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
    The Linux-only timeouts are also observed in the integrity testing
results.
    JIRA HARMONY-5833 has been filed for one of them.

 6) JettyScenario:
    The Linux x86 running failed because of the unresolved HARMONY-5219.

 7) Reliability:
    Several failures are observed in windows x86_64 running.
    7.1) Old regressions
         api.net.DatagramTest (HARMONY-5531)
         api.text.DecimalFormat_Locales - (in M6 it is recorded as also
intermittent in M5)

    7.2) New/intemittent regressions
         api.kernel.thread.ThreadLocalTest.ThreadLocalTest
         api.kernel.exec.RunExec

 8) Stress
    Different test cases failed on different platforms.
    8.1) Timeouts on Linux x86.

stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003

stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007

stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009

stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011

    8.2) Failed cases on Windows x86_64 with unknown reasons.

stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2

stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses

    I have not found records for them.

 9) Strut_test
    Broken with same error report as M6.

 10) Eclipse Hello World Application.
    Although the testing framework just reports EUT-API status as "PASSED".
A fresh workspace running just failed in configuration stage (
http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e)
since SVN commit 641928 (which committed patch for HARMONY-4569).

thanks,
chunrong

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message