harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nathan Beyer" <ndbe...@apache.org>
Subject Re: [testing] first M7 candidate (r681495) testing status
Date Tue, 12 Aug 2008 01:40:27 GMT
What's being used to produce the integrity tests? I have an x86_64 Linux box
that I can dedicate to testing. I've tried just using buildtest with
'classlib,drlvm,classib-test,drlvm-test', but I haven't gotten a clean pass
yet.

-Nathan

On Mon, Aug 11, 2008 at 12:26 PM, chunrong lai <chunronglai@gmail.com>wrote:

> Hi, all:
>
>  Here is r681495 snapshot testing status:
> http://people.apache.org/~chunrong/snapshots/r681495/index.html<http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>.
> I am using
> two platforms (Linux x86, windows x86_64) at the moment. Hopefully we will
> have other two platforms in future for M8. Well, although we are testing
> only two platforms for M7, my experience is that the status for another two
> platforms should be not worse or just include some extra intermittent
> errors
> which can be investigated in some later stages.
>
>  The following suites passed on Linux x86/Windows x86_64 platforms: Ant
> Scenario (or self-hosting), Axis application, Dacapo, DRLVM regression
> tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS VM Test Suite.
>
>  Most of the failures are known issues (for M6). Although we can observe
> 15~20 new issues, those issues happen only in 1 platform and they look more
> like the intermittent/timeout issues (less reproducible) to me. Overall I
> myself would like to think r681495 is more stable than M6.
>
>  Please add your comments and clarifications (please also see M6 testing
> results http://people.apache.org/~smishura/r653525/<http://people.apache.org/%7Esmishura/r653525/>
> ,
>
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand
> integrity testing results
> http://people.apache.org/~chunrong/harmony-integrity/<http://people.apache.org/%7Echunrong/harmony-integrity/>as
a comparison).
>
>  1) Classlib:
>    1.1) Since r644719 (which committed patch for HARMONY-5684)
>
>        org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest
>
>
> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
>
>         failed in both platforms
>
>    1.2) Two failures
>
>        org.apache.harmony.luni.tests.java.net.MulticastSocketTest (Failed
> in windows_x86 running of M6)
>        tests.api.java.security.PermissionCollectionTest           (Failed
> in M6)
>
>         are observed in Linux x86.
>
>  2) DRLVM tests:
>    2.1) One failure
>
>         java.lang.ClassGenericsTest.test_2
>
>         is observed in Linux x86 snapshot testing.
>         I can see some old discussion in the mailing list about that but I
> am not sure the expected status here.
>         They should be intemittent errors since the integrity testing just
> run well mostly.
>
>  3) EUTs:
>
>    3.1) Linux x86: 99.36%
>         A recorded JIRA for this suite is HARMONY-2914 which wastes file
> handlers and makes system unstable.
>
>  4) Functional:
>    4.1) Old regressions on both platforms:
>         api.java.text.MessageFormat (HARMONY-5430)
>         api.java.util.jar.Manifest  (HARMONY-5473)
>         api.java.beans.beancontext.BeanContextTest (also in M6, recorded
> as  regression caused by changes in locale data)
>         api.java.beans.persistence.EncoderTest  (also in M6, recorded as
> regression in the beans module)
>         api.java.beans.persistence.EncoderDecoderTest (also in M6,
> regression in the beans module)
>         reg.vm.btest5625 (also in M6, recorded as intermittent and not
> reproducible manually)
>
>    4.2) Old regressions on 1 platform
>         api.java.rmi.basicexception (ERROR in Linux x86, HARMONY-5823)
>         api.java.rmi.basicregistry.RemoteServerTest (ERROR in Linux x86,
> HARMONY-5823)
>         jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows x86_64, in M6 it
> is recorded as regression since M4)
>         java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR on Linux x86,
> recorded as Timeout, not reproducible
>  in M6)
>         reg.vm.btest5717 (ERROR on Windows X86_64, recorded as "timeout,
> the test is too heavy" in M6)
>         jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1 (FAILED in
> windows x86_64, recorded as "looks like
> issue in test" in M6)
>         jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on windows x86_64, in
> M6 it is recorded as not regression and started to fail on M5)
>         reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64, recorded also
> failed on M3 and might be issue with the test)
>
>    4.3) New regressions on 1 platform (looks intermittent)
>         reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
>         func.reg.jit.btest5710.Btest5710 (FAILED in Linux x86)
>         api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06 (ERROR
> in Linux x86)
>         api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05 (ERROR
> in Linux x86)
>
>  5) JDKTools Tests:
>    Several timeouts are observed in Linux x86 snapshot running. They are:
>
>
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001
>
>
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002
>
>
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003
>
>
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004
>
>
> org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04
>
>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001
>
>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001
>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume
>
>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001
>
>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
>    The Linux-only timeouts are also observed in the integrity testing
> results.
>    JIRA HARMONY-5833 has been filed for one of them.
>
>  6) JettyScenario:
>    The Linux x86 running failed because of the unresolved HARMONY-5219.
>
>  7) Reliability:
>    Several failures are observed in windows x86_64 running.
>    7.1) Old regressions
>         api.net.DatagramTest (HARMONY-5531)
>         api.text.DecimalFormat_Locales - (in M6 it is recorded as also
> intermittent in M5)
>
>    7.2) New/intemittent regressions
>         api.kernel.thread.ThreadLocalTest.ThreadLocalTest
>         api.kernel.exec.RunExec
>
>  8) Stress
>    Different test cases failed on different platforms.
>    8.1) Timeouts on Linux x86.
>
>
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003
>
>
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007
>
>
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009
>
>
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011
>
>    8.2) Failed cases on Windows x86_64 with unknown reasons.
>
>
> stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2
>
>
> stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses
>
>    I have not found records for them.
>
>  9) Strut_test
>    Broken with same error report as M6.
>
>  10) Eclipse Hello World Application.
>    Although the testing framework just reports EUT-API status as "PASSED".
> A fresh workspace running just failed in configuration stage (
>
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e
> )
> since SVN commit 641928 (which committed patch for HARMONY-4569).
>
> thanks,
> chunrong
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message