Mailing-List: contact dev-help@harmony.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@harmony.apache.org
Received-SPF: pass (athena.apache.org: domain of sianjanuary@googlemail.com
 designates 209.85.198.251 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=googlemail.com; s=gamma;
        h=message-id:date:from:to:subject:in-reply-to:mime-version
         :content-type:content-transfer-encoding:content-disposition
         :references;
        b=P1+MS44S4WrzZ77Wgtt5YPyLa+9Xb5zy+/GB2vD/OOWRNHjNFK0jx3hOKDmclejNsm
         Um7gMNMZZsd/wiMivl9/WVYdLgeS4IKpkmyJ4j0IH6t0+yJhb3LdKEaRRFQDcom3/mpG
         q135Bqq9XZ7Uqlw1sAmUg26cq7gXclcD9RIJU=
Message-ID: <fc2fc95c0808150615k454ac96dkba59f30eaee47348@mail.gmail.com>
Date: Fri, 15 Aug 2008 14:15:08 +0100
From: "Sian January" <sianjanuary@googlemail.com>
To: dev@harmony.apache.org
Subject: Re: [testing] first M7 candidate (r681495) testing status
In-Reply-To: <f9761b910808141900r30777763yad049a6885a87168@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
References: <f9761b910808111026l28811b1bo708ea7896b2c965d@mail.gmail.com>
	 <fc2fc95c0808140523j3fef2986w1360fe7c8bee2f46@mail.gmail.com>
	 <f9761b910808141900r30777763yad049a6885a87168@mail.gmail.com>

On 15/08/2008, chunrong lai <chunronglai@gmail.com> wrote:
>  I think none of these are blocker for M7 because they are basically old
> issues or intermettent errors. I also think that we are doing better than
> M6.

Ok - thanks.  I'm going to kick off a vote now unless anyone objects
in the next few minutes.  I think 2 and a half weeks should be long
enough for everyone to have done the testing they intended to do by
now.

>
>  It is also noted that (as the comment in HARMONY-5747) the error in 10) is
> a new issue due to an old classlib checkin r641928 which can be stably
> reproduced . Maybe we need to have a look in it?
>

Investigations in HARMONY-5747 found that the other problem was an
Eclipse bug.  Eclipse definitely still runs with Harmony, so although
I do think we should investigate this I don't think it's a blocker for
M7.

> On 8/14/08, Sian January <sianjanuary@googlemail.com> wrote:
> >
> > Does anyone have a feel for whether any of these failures should be
> > blockers for M7?  In terms of the Harmony test suite I think the pass
> > rate is fine because we're doing better than M6, and EUTs look ok too,
> > but I'm not really sure if we need to be concerned about any of:
> >
> > 4) Functional
> > 5) JDKTools Tests
> > 6) JettyScenario
> > 7) Reliability
> > 8) Stress
> >
> > or
> >
> > 10) Eclipse Hello World Application.
> >
> > Any thoughts?
> >
> > If none of these are blockers then it would be good to start a vote in
> > the next day or two because the code has been frozen for nearly 3
> > weeks now.
> >
> > Thanks,
> >
> > Sian
> >
> >
> > On 11/08/2008, chunrong lai <chunronglai@gmail.com> wrote:
> > > Hi, all:
> > >
> > >  Here is r681495 snapshot testing status:
> > > http://people.apache.org/~chunrong/snapshots/r681495/index.html. I am
> > using
> > > two platforms (Linux x86, windows x86_64) at the moment. Hopefully we
> > will
> > > have other two platforms in future for M8. Well, although we are testing
> > > only two platforms for M7, my experience is that the status for another
> > two
> > > platforms should be not worse or just include some extra intermittent
> > errors
> > > which can be investigated in some later stages.
> > >
> > >  The following suites passed on Linux x86/Windows x86_64 platforms: Ant
> > > Scenario (or self-hosting), Axis application, Dacapo, DRLVM regression
> > > tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS VM Test Suite.
> > >
> > >  Most of the failures are known issues (for M6). Although we can observe
> > > 15~20 new issues, those issues happen only in 1 platform and they look
> > more
> > > like the intermittent/timeout issues (less reproducible) to me. Overall I
> > > myself would like to think r681495 is more stable than M6.
> > >
> > >  Please add your comments and clarifications (please also see M6 testing
> > > results http://people.apache.org/~smishura/r653525/,
> > >
> > http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand
> > > integrity testing results
> > > http://people.apache.org/~chunrong/harmony-integrity/ as a comparison).
> > >
> > >  1) Classlib:
> > >    1.1) Since r644719 (which committed patch for HARMONY-5684)
> > >
> > >        org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest
> > >
> > >
> > org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
> > >
> > >         failed in both platforms
> > >
> > >    1.2) Two failures
> > >
> > >        org.apache.harmony.luni.tests.java.net.MulticastSocketTest (Failed
> > > in windows_x86 running of M6)
> > >        tests.api.java.security.PermissionCollectionTest           (Failed
> > > in M6)
> > >
> > >         are observed in Linux x86.
> > >
> > >  2) DRLVM tests:
> > >    2.1) One failure
> > >
> > >         java.lang.ClassGenericsTest.test_2
> > >
> > >         is observed in Linux x86 snapshot testing.
> > >         I can see some old discussion in the mailing list about that but
> > I
> > > am not sure the expected status here.
> > >         They should be intemittent errors since the integrity testing
> > just
> > > run well mostly.
> > >
> > >  3) EUTs:
> > >
> > >    3.1) Linux x86: 99.36%
> > >         A recorded JIRA for this suite is HARMONY-2914 which wastes file
> > > handlers and makes system unstable.
> > >
> > >  4) Functional:
> > >    4.1) Old regressions on both platforms:
> > >         api.java.text.MessageFormat (HARMONY-5430)
> > >         api.java.util.jar.Manifest  (HARMONY-5473)
> > >         api.java.beans.beancontext.BeanContextTest (also in M6, recorded
> > > as  regression caused by changes in locale data)
> > >         api.java.beans.persistence.EncoderTest  (also in M6, recorded as
> > > regression in the beans module)
> > >         api.java.beans.persistence.EncoderDecoderTest (also in M6,
> > > regression in the beans module)
> > >         reg.vm.btest5625 (also in M6, recorded as intermittent and not
> > > reproducible manually)
> > >
> > >    4.2) Old regressions on 1 platform
> > >         api.java.rmi.basicexception (ERROR in Linux x86, HARMONY-5823)
> > >         api.java.rmi.basicregistry.RemoteServerTest (ERROR in Linux x86,
> > > HARMONY-5823)
> > >         jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows x86_64, in M6
> > it
> > > is recorded as regression since M4)
> > >         java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR on Linux x86,
> > > recorded as Timeout, not reproducible
> > >  in M6)
> > >         reg.vm.btest5717 (ERROR on Windows X86_64, recorded as "timeout,
> > > the test is too heavy" in M6)
> > >         jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1 (FAILED in
> > > windows x86_64, recorded as "looks like
> > > issue in test" in M6)
> > >         jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on windows x86_64,
> > in
> > > M6 it is recorded as not regression and started to fail on M5)
> > >         reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64, recorded
> > also
> > > failed on M3 and might be issue with the test)
> > >
> > >    4.3) New regressions on 1 platform (looks intermittent)
> > >         reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
> > >         func.reg.jit.btest5710.Btest5710 (FAILED in Linux x86)
> > >         api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06 (ERROR
> > > in Linux x86)
> > >         api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05 (ERROR
> > > in Linux x86)
> > >
> > >  5) JDKTools Tests:
> > >    Several timeouts are observed in Linux x86 snapshot running. They are:
> > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001
> > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002
> > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003
> > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004
> > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04
> > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001
> > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001
> > >
> > > org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume
> > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001
> > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
> > >    The Linux-only timeouts are also observed in the integrity testing
> > > results.
> > >    JIRA HARMONY-5833 has been filed for one of them.
> > >
> > >  6) JettyScenario:
> > >    The Linux x86 running failed because of the unresolved HARMONY-5219.
> > >
> > >  7) Reliability:
> > >    Several failures are observed in windows x86_64 running.
> > >    7.1) Old regressions
> > >         api.net.DatagramTest (HARMONY-5531)
> > >         api.text.DecimalFormat_Locales - (in M6 it is recorded as also
> > > intermittent in M5)
> > >
> > >    7.2) New/intemittent regressions
> > >         api.kernel.thread.ThreadLocalTest.ThreadLocalTest
> > >         api.kernel.exec.RunExec
> > >
> > >  8) Stress
> > >    Different test cases failed on different platforms.
> > >    8.1) Timeouts on Linux x86.
> > >
> > >
> > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003
> > >
> > >
> > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007
> > >
> > >
> > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009
> > >
> > >
> > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011
> > >
> > >    8.2) Failed cases on Windows x86_64 with unknown reasons.
> > >
> > >
> > stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2
> > >
> > >
> > stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses
> > >
> > >    I have not found records for them.
> > >
> > >  9) Strut_test
> > >    Broken with same error report as M6.
> > >
> > >  10) Eclipse Hello World Application.
> > >    Although the testing framework just reports EUT-API status as
> > "PASSED".
> > > A fresh workspace running just failed in configuration stage (
> > >
> > http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e
> > )
> > > since SVN commit 641928 (which committed patch for HARMONY-4569).
> > >
> > > thanks,
> > > chunrong
> > >
> >
> >
> > --
> > Unless stated otherwise above:
> > IBM United Kingdom Limited - Registered in England and Wales with number
> > 741598.
> > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
> >
>


-- 
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU