Mailing-List: contact dev-help@harmony.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@harmony.apache.org
Received-SPF: pass (athena.apache.org: domain of chunronglai@gmail.com
 designates 74.125.44.157 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:date:from:to:subject:in-reply-to:mime-version
         :content-type:references;
        b=lEqCLsMZzc2CPXVXWdAyBtdUX79+kvoHZE9+2Y7tKaPOrKYOMAf9+vOEem7Nt2o9bu
         z+QONmc+EpBBtoVsxmMq7AqpLjJ07ErjOHH/XXOfQ7kXLWQ78ANEfw4ff7mPe0XCJnDf
         j827w51oxMi4hi0UDVpkpQbpmf4NRaStOJ358=
Message-ID: <f9761b910808131918s731f9fecn9f751cfef4dbb3d2@mail.gmail.com>
Date: Thu, 14 Aug 2008 10:18:14 +0800
From: "chunrong lai" <chunronglai@gmail.com>
To: dev@harmony.apache.org
Subject: Re: [testing] first M7 candidate (r681495) testing status
In-Reply-To: <3b3f27c60808131909v582797c8u56bbb30c958113d6@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_14533_29305447.1218680294164"
References: <f9761b910808111026l28811b1bo708ea7896b2c965d@mail.gmail.com>
	 <3b3f27c60808111840q6a4a9a18i23ccd202d9a00f33@mail.gmail.com>
	 <f9761b910808130121p20901242o7fa267e9dbe4ca5f@mail.gmail.com>
	 <3b3f27c60808131736la3e72exb7d29483fcda17ab@mail.gmail.com>
	 <f9761b910808131858x1df19977id120d3cb4024e535@mail.gmail.com>
	 <3b3f27c60808131909v582797c8u56bbb30c958113d6@mail.gmail.com>

------=_Part_14533_29305447.1218680294164
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

 I understand that CruiseControl does trigger the test suite running. But
some test suites themselves (classlib, dacapo etc) do require GUI support.
So in Linux one needs to run cruisecontrol ("ant run-cc") in an environment
with X-connection (for example, in X, vnc-client) instead of text-mode
client (putty, ssh, term etc).


On 8/14/08, Nathan Beyer <ndbeyer@apache.org> wrote:
>
> On Wed, Aug 13, 2008 at 8:58 PM, chunrong lai <chunronglai@gmail.com>
> wrote:
> > (1) M7 status is got in Fedora distros and M6/M5 status were got in SuSe
> > Linux distros.
> > (2) According to the stacktrace
> > ( java.awt.Toolkit.init,  java.awt.SystemColor.<init>) of the
> > java.beans.Encoder error (I have not observed such error reports in
> previous
> > logs),  I guess the problem is from GUI system initialization. Is
> > X-connection available in your testing? In my understanding that is
> required
> > to run classlib test.
>
> It's running under CruiseControl, like the howto suggested, how do you
> do an X-connection with that?
>
> >
> >
> > On 8/14/08, Nathan Beyer <ndbeyer@apache.org> wrote:
> >>
> >> What Linux distro is it? I'm running Ubuntu and I'm seeing some
> >> different issues.
> >>
> >> I'm getting this failure -
> >>
> >> java.lang.ExceptionInInitializerError
> >> at java.lang.reflect.VMField.getObject(VMField.java)
> >> at java.lang.reflect.Field.get(Field.java:163)
> >> at
> >>
> java.beans.StaticFieldPersistenceDelegate.init(StaticFieldPersistenceDelegate.java:36)
> >> at
> >>
> java.beans.AwtSystemColorPersistenceDelegate.<clinit>(AwtSystemColorPersistenceDelegate.java:28)
> >> at java.beans.Encoder.<clinit>(Encoder.java:124)
> >> at
> >>
> java.beans.ArrayPersistenceDelegateTest.testInstantiate_Normal(ArrayPersistenceDelegateTest.java:130)
> >> at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java)
> >> Caused by: java.lang.NullPointerException
> >> at java.awt.Toolkit.init(Toolkit.java:566)
> >> at java.awt.Toolkit.<init>(Toolkit.java:540)
> >> at java.awt.ToolkitImpl.<init>(ToolkitImpl.java:40)
> >> at java.awt.Toolkit.getDefaultToolkit(Toolkit.java:472)
> >> at java.awt.SystemColor.<init>(SystemColor.java:143)
> >> at java.awt.SystemColor.<clinit>(SystemColor.java:91)
> >> at java.lang.reflect.VMField.getObject(VMField.java)
> >> ... 18 more
> >>
> >> Then a bunch of NoClassDefFoundErrors for java.beans.Encoder.
> >>
> >>
> >> On Wed, Aug 13, 2008 at 3:21 AM, chunrong lai <chunronglai@gmail.com>
> >> wrote:
> >> >
> >> > hi, Nathan:
> >> >     It is also noted that currently we do setup Linux x86-64 integrity
> >> > testing, which eventually test every SVN revision, with results
> reported
> >> in
> >> > http://people.apache.org/~chunrong/harmony-integrity/. Your mentioned
> >> test
> >> > suites classib-test,drlvm-test are included there. (classlib+drlvm
> >> > are almost same as the HDK build reported in the page).
> >> >     We only use two platforms currently for the snapshot testing
> cycle,
> >> > (table is generated by
> >> >
> >>
> http://svn.apache.org/viewvc/harmony/enhanced/buildtest/trunk/tools/snapshot/snapshots-table/
> >> )
> >> > , to test more test suites with longer time for every snapshot. In my
> >> > observation adding two other platforms here just does not necessarily
> add
> >> > enough value in current stage of Harmony.
> >> >     I do think more testings/discussions help to improve Harmony
> quality.
> >> > Thanks for the dedication. If there are more I can help please let me
> >> know.
> >> >
> >> > On 8/12/08, Nathan Beyer <ndbeyer@apache.org> wrote:
> >> > >
> >> > > What's being used to produce the integrity tests? I have an x86_64
> >> Linux
> >> > > box
> >> > > that I can dedicate to testing. I've tried just using buildtest with
> >> > > 'classlib,drlvm,classib-test,drlvm-test', but I haven't gotten a
> clean
> >> pass
> >> > > yet.
> >> > >
> >> > > -Nathan
> >> > >
> >> > > On Mon, Aug 11, 2008 at 12:26 PM, chunrong lai <
> chunronglai@gmail.com
> >> > > >wrote:
> >> > >
> >> > > > Hi, all:
> >> > > >
> >> > > >  Here is r681495 snapshot testing status:
> >> > > > http://people.apache.org/~chunrong/snapshots/r681495/index.html<
> >> > > http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>.
> >> > > > I am using
> >> > > > two platforms (Linux x86, windows x86_64) at the moment. Hopefully
> we
> >> > > will
> >> > > > have other two platforms in future for M8. Well, although we are
> >> testing
> >> > > > only two platforms for M7, my experience is that the status for
> >> another
> >> > > two
> >> > > > platforms should be not worse or just include some extra
> intermittent
> >> > > > errors
> >> > > > which can be investigated in some later stages.
> >> > > >
> >> > > >  The following suites passed on Linux x86/Windows x86_64
> platforms:
> >> Ant
> >> > > > Scenario (or self-hosting), Axis application, Dacapo, DRLVM
> >> regression
> >> > > > tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS VM Test
> >> Suite.
> >> > > >
> >> > > >  Most of the failures are known issues (for M6). Although we can
> >> observe
> >> > > > 15~20 new issues, those issues happen only in 1 platform and they
> >> look
> >> > > more
> >> > > > like the intermittent/timeout issues (less reproducible) to me.
> >> Overall I
> >> > > > myself would like to think r681495 is more stable than M6.
> >> > > >
> >> > > >  Please add your comments and clarifications (please also see M6
> >> testing
> >> > > > results http://people.apache.org/~smishura/r653525/<
> >> > > http://people.apache.org/%7Esmishura/r653525/>
> >> > > > ,
> >> > > >
> >> > > >
> >> > >
> >>
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand
> >> > > > integrity testing results
> >> > > > http://people.apache.org/~chunrong/harmony-integrity/<
> >> > > http://people.apache.org/%7Echunrong/harmony-integrity/>as a
> >> comparison).
> >> > > >
> >> > > >  1) Classlib:
> >> > > >    1.1) Since r644719 (which committed patch for HARMONY-5684)
> >> > > >
> >> > > >        org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
> >> > > >
> >> > > >         failed in both platforms
> >> > > >
> >> > > >    1.2) Two failures
> >> > > >
> >> > > >        org.apache.harmony.luni.tests.java.net.MulticastSocketTest
> >> (Failed
> >> > > > in windows_x86 running of M6)
> >> > > >        tests.api.java.security.PermissionCollectionTest
> >> (Failed
> >> > > > in M6)
> >> > > >
> >> > > >         are observed in Linux x86.
> >> > > >
> >> > > >  2) DRLVM tests:
> >> > > >    2.1) One failure
> >> > > >
> >> > > >         java.lang.ClassGenericsTest.test_2
> >> > > >
> >> > > >         is observed in Linux x86 snapshot testing.
> >> > > >         I can see some old discussion in the mailing list about
> that
> >> but
> >> > > I
> >> > > > am not sure the expected status here.
> >> > > >         They should be intemittent errors since the integrity
> testing
> >> > > just
> >> > > > run well mostly.
> >> > > >
> >> > > >  3) EUTs:
> >> > > >
> >> > > >    3.1) Linux x86: 99.36%
> >> > > >         A recorded JIRA for this suite is HARMONY-2914 which
> wastes
> >> file
> >> > > > handlers and makes system unstable.
> >> > > >
> >> > > >  4) Functional:
> >> > > >    4.1) Old regressions on both platforms:
> >> > > >         api.java.text.MessageFormat (HARMONY-5430)
> >> > > >         api.java.util.jar.Manifest  (HARMONY-5473)
> >> > > >         api.java.beans.beancontext.BeanContextTest (also in M6,
> >> recorded
> >> > > > as  regression caused by changes in locale data)
> >> > > >         api.java.beans.persistence.EncoderTest  (also in M6,
> recorded
> >> as
> >> > > > regression in the beans module)
> >> > > >         api.java.beans.persistence.EncoderDecoderTest (also in M6,
> >> > > > regression in the beans module)
> >> > > >         reg.vm.btest5625 (also in M6, recorded as intermittent and
> >> not
> >> > > > reproducible manually)
> >> > > >
> >> > > >    4.2) Old regressions on 1 platform
> >> > > >         api.java.rmi.basicexception (ERROR in Linux x86,
> >> HARMONY-5823)
> >> > > >         api.java.rmi.basicregistry.RemoteServerTest (ERROR in
> Linux
> >> x86,
> >> > > > HARMONY-5823)
> >> > > >         jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows x86_64,
> in
> >> M6
> >> > > it
> >> > > > is recorded as regression since M4)
> >> > > >         java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR on
> Linux
> >> x86,
> >> > > > recorded as Timeout, not reproducible
> >> > > >  in M6)
> >> > > >         reg.vm.btest5717 (ERROR on Windows X86_64, recorded as
> >> "timeout,
> >> > > > the test is too heavy" in M6)
> >> > > >         jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1
> (FAILED
> >> in
> >> > > > windows x86_64, recorded as "looks like
> >> > > > issue in test" in M6)
> >> > > >         jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on windows
> >> x86_64,
> >> > > in
> >> > > > M6 it is recorded as not regression and started to fail on M5)
> >> > > >         reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64,
> recorded
> >> > > also
> >> > > > failed on M3 and might be issue with the test)
> >> > > >
> >> > > >    4.3) New regressions on 1 platform (looks intermittent)
> >> > > >         reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
> >> > > >         func.reg.jit.btest5710.Btest5710 (FAILED in Linux x86)
> >> > > >         api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06
> >> (ERROR
> >> > > > in Linux x86)
> >> > > >         api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05
> >> (ERROR
> >> > > > in Linux x86)
> >> > > >
> >> > > >  5) JDKTools Tests:
> >> > > >    Several timeouts are observed in Linux x86 snapshot running.
> They
> >> are:
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001
> >> > > >
> >> > > >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
> >> > > >    The Linux-only timeouts are also observed in the integrity
> testing
> >> > > > results.
> >> > > >    JIRA HARMONY-5833 has been filed for one of them.
> >> > > >
> >> > > >  6) JettyScenario:
> >> > > >    The Linux x86 running failed because of the unresolved
> >> HARMONY-5219.
> >> > > >
> >> > > >  7) Reliability:
> >> > > >    Several failures are observed in windows x86_64 running.
> >> > > >    7.1) Old regressions
> >> > > >         api.net.DatagramTest (HARMONY-5531)
> >> > > >         api.text.DecimalFormat_Locales - (in M6 it is recorded as
> >> also
> >> > > > intermittent in M5)
> >> > > >
> >> > > >    7.2) New/intemittent regressions
> >> > > >         api.kernel.thread.ThreadLocalTest.ThreadLocalTest
> >> > > >         api.kernel.exec.RunExec
> >> > > >
> >> > > >  8) Stress
> >> > > >    Different test cases failed on different platforms.
> >> > > >    8.1) Timeouts on Linux x86.
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011
> >> > > >
> >> > > >    8.2) Failed cases on Windows x86_64 with unknown reasons.
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses
> >> > > >
> >> > > >    I have not found records for them.
> >> > > >
> >> > > >  9) Strut_test
> >> > > >    Broken with same error report as M6.
> >> > > >
> >> > > >  10) Eclipse Hello World Application.
> >> > > >    Although the testing framework just reports EUT-API status as
> >> > > "PASSED".
> >> > > > A fresh workspace running just failed in configuration stage (
> >> > > >
> >> > > >
> >> > >
> >>
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e
> >> > > > )
> >> > > > since SVN commit 641928 (which committed patch for HARMONY-4569).
> >> > > >
> >> > > > thanks,
> >> > > > chunrong
> >> > > >
> >> > >
> >>
> >
>

------=_Part_14533_29305447.1218680294164--