harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nathan Beyer" <nbe...@gmail.com>
Subject Re: [testing] first M7 candidate (r681495) testing status
Date Thu, 14 Aug 2008 02:42:08 GMT
I understand the concept, but running CruiseControl (a headless daemon
process) and a local X session are a bit at odds, aren't they?

I'm asking, in practical terms, how are you doing it? Start a local X
session, start an xterm, invoke cruise control and let it go? Is there
anyway to do this remotely? Direct access to a machine isn't always an
option and generally a remote client can't be permanently left up. Is
there anyway to do a 'nohup' with an X session?

-Nathan

On Wed, Aug 13, 2008 at 9:18 PM, chunrong lai <chunronglai@gmail.com> wrote:
>  I understand that CruiseControl does trigger the test suite running. But
> some test suites themselves (classlib, dacapo etc) do require GUI support.
> So in Linux one needs to run cruisecontrol ("ant run-cc") in an environment
> with X-connection (for example, in X, vnc-client) instead of text-mode
> client (putty, ssh, term etc).
>
>
>
> On 8/14/08, Nathan Beyer <ndbeyer@apache.org> wrote:
>>
>> On Wed, Aug 13, 2008 at 8:58 PM, chunrong lai <chunronglai@gmail.com>
>> wrote:
>> > (1) M7 status is got in Fedora distros and M6/M5 status were got in SuSe
>> > Linux distros.
>> > (2) According to the stacktrace
>> > ( java.awt.Toolkit.init,  java.awt.SystemColor.<init>) of the
>> > java.beans.Encoder error (I have not observed such error reports in
>> previous
>> > logs),  I guess the problem is from GUI system initialization. Is
>> > X-connection available in your testing? In my understanding that is
>> required
>> > to run classlib test.
>>
>> It's running under CruiseControl, like the howto suggested, how do you
>> do an X-connection with that?
>>
>> >
>> >
>> > On 8/14/08, Nathan Beyer <ndbeyer@apache.org> wrote:
>> >>
>> >> What Linux distro is it? I'm running Ubuntu and I'm seeing some
>> >> different issues.
>> >>
>> >> I'm getting this failure -
>> >>
>> >> java.lang.ExceptionInInitializerError
>> >> at java.lang.reflect.VMField.getObject(VMField.java)
>> >> at java.lang.reflect.Field.get(Field.java:163)
>> >> at
>> >>
>> java.beans.StaticFieldPersistenceDelegate.init(StaticFieldPersistenceDelegate.java:36)
>> >> at
>> >>
>> java.beans.AwtSystemColorPersistenceDelegate.<clinit>(AwtSystemColorPersistenceDelegate.java:28)
>> >> at java.beans.Encoder.<clinit>(Encoder.java:124)
>> >> at
>> >>
>> java.beans.ArrayPersistenceDelegateTest.testInstantiate_Normal(ArrayPersistenceDelegateTest.java:130)
>> >> at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java)
>> >> Caused by: java.lang.NullPointerException
>> >> at java.awt.Toolkit.init(Toolkit.java:566)
>> >> at java.awt.Toolkit.<init>(Toolkit.java:540)
>> >> at java.awt.ToolkitImpl.<init>(ToolkitImpl.java:40)
>> >> at java.awt.Toolkit.getDefaultToolkit(Toolkit.java:472)
>> >> at java.awt.SystemColor.<init>(SystemColor.java:143)
>> >> at java.awt.SystemColor.<clinit>(SystemColor.java:91)
>> >> at java.lang.reflect.VMField.getObject(VMField.java)
>> >> ... 18 more
>> >>
>> >> Then a bunch of NoClassDefFoundErrors for java.beans.Encoder.
>> >>
>> >>
>> >> On Wed, Aug 13, 2008 at 3:21 AM, chunrong lai <chunronglai@gmail.com>
>> >> wrote:
>> >> >
>> >> > hi, Nathan:
>> >> >     It is also noted that currently we do setup Linux x86-64 integrity
>> >> > testing, which eventually test every SVN revision, with results
>> reported
>> >> in
>> >> > http://people.apache.org/~chunrong/harmony-integrity/. Your mentioned
>> >> test
>> >> > suites classib-test,drlvm-test are included there. (classlib+drlvm
>> >> > are almost same as the HDK build reported in the page).
>> >> >     We only use two platforms currently for the snapshot testing
>> cycle,
>> >> > (table is generated by
>> >> >
>> >>
>> http://svn.apache.org/viewvc/harmony/enhanced/buildtest/trunk/tools/snapshot/snapshots-table/
>> >> )
>> >> > , to test more test suites with longer time for every snapshot. In
my
>> >> > observation adding two other platforms here just does not necessarily
>> add
>> >> > enough value in current stage of Harmony.
>> >> >     I do think more testings/discussions help to improve Harmony
>> quality.
>> >> > Thanks for the dedication. If there are more I can help please let
me
>> >> know.
>> >> >
>> >> > On 8/12/08, Nathan Beyer <ndbeyer@apache.org> wrote:
>> >> > >
>> >> > > What's being used to produce the integrity tests? I have an x86_64
>> >> Linux
>> >> > > box
>> >> > > that I can dedicate to testing. I've tried just using buildtest
with
>> >> > > 'classlib,drlvm,classib-test,drlvm-test', but I haven't gotten
a
>> clean
>> >> pass
>> >> > > yet.
>> >> > >
>> >> > > -Nathan
>> >> > >
>> >> > > On Mon, Aug 11, 2008 at 12:26 PM, chunrong lai <
>> chunronglai@gmail.com
>> >> > > >wrote:
>> >> > >
>> >> > > > Hi, all:
>> >> > > >
>> >> > > >  Here is r681495 snapshot testing status:
>> >> > > > http://people.apache.org/~chunrong/snapshots/r681495/index.html<
>> >> > > http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>.
>> >> > > > I am using
>> >> > > > two platforms (Linux x86, windows x86_64) at the moment.
Hopefully
>> we
>> >> > > will
>> >> > > > have other two platforms in future for M8. Well, although
we are
>> >> testing
>> >> > > > only two platforms for M7, my experience is that the status
for
>> >> another
>> >> > > two
>> >> > > > platforms should be not worse or just include some extra
>> intermittent
>> >> > > > errors
>> >> > > > which can be investigated in some later stages.
>> >> > > >
>> >> > > >  The following suites passed on Linux x86/Windows x86_64
>> platforms:
>> >> Ant
>> >> > > > Scenario (or self-hosting), Axis application, Dacapo, DRLVM
>> >> regression
>> >> > > > tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS
VM Test
>> >> Suite.
>> >> > > >
>> >> > > >  Most of the failures are known issues (for M6). Although
we can
>> >> observe
>> >> > > > 15~20 new issues, those issues happen only in 1 platform
and they
>> >> look
>> >> > > more
>> >> > > > like the intermittent/timeout issues (less reproducible)
to me.
>> >> Overall I
>> >> > > > myself would like to think r681495 is more stable than M6.
>> >> > > >
>> >> > > >  Please add your comments and clarifications (please also
see M6
>> >> testing
>> >> > > > results http://people.apache.org/~smishura/r653525/<
>> >> > > http://people.apache.org/%7Esmishura/r653525/>
>> >> > > > ,
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand
>> >> > > > integrity testing results
>> >> > > > http://people.apache.org/~chunrong/harmony-integrity/<
>> >> > > http://people.apache.org/%7Echunrong/harmony-integrity/>as
a
>> >> comparison).
>> >> > > >
>> >> > > >  1) Classlib:
>> >> > > >    1.1) Since r644719 (which committed patch for HARMONY-5684)
>> >> > > >
>> >> > > >        org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
>> >> > > >
>> >> > > >         failed in both platforms
>> >> > > >
>> >> > > >    1.2) Two failures
>> >> > > >
>> >> > > >        org.apache.harmony.luni.tests.java.net.MulticastSocketTest
>> >> (Failed
>> >> > > > in windows_x86 running of M6)
>> >> > > >        tests.api.java.security.PermissionCollectionTest
>> >> (Failed
>> >> > > > in M6)
>> >> > > >
>> >> > > >         are observed in Linux x86.
>> >> > > >
>> >> > > >  2) DRLVM tests:
>> >> > > >    2.1) One failure
>> >> > > >
>> >> > > >         java.lang.ClassGenericsTest.test_2
>> >> > > >
>> >> > > >         is observed in Linux x86 snapshot testing.
>> >> > > >         I can see some old discussion in the mailing list
about
>> that
>> >> but
>> >> > > I
>> >> > > > am not sure the expected status here.
>> >> > > >         They should be intemittent errors since the integrity
>> testing
>> >> > > just
>> >> > > > run well mostly.
>> >> > > >
>> >> > > >  3) EUTs:
>> >> > > >
>> >> > > >    3.1) Linux x86: 99.36%
>> >> > > >         A recorded JIRA for this suite is HARMONY-2914 which
>> wastes
>> >> file
>> >> > > > handlers and makes system unstable.
>> >> > > >
>> >> > > >  4) Functional:
>> >> > > >    4.1) Old regressions on both platforms:
>> >> > > >         api.java.text.MessageFormat (HARMONY-5430)
>> >> > > >         api.java.util.jar.Manifest  (HARMONY-5473)
>> >> > > >         api.java.beans.beancontext.BeanContextTest (also
in M6,
>> >> recorded
>> >> > > > as  regression caused by changes in locale data)
>> >> > > >         api.java.beans.persistence.EncoderTest  (also in
M6,
>> recorded
>> >> as
>> >> > > > regression in the beans module)
>> >> > > >         api.java.beans.persistence.EncoderDecoderTest (also
in M6,
>> >> > > > regression in the beans module)
>> >> > > >         reg.vm.btest5625 (also in M6, recorded as intermittent
and
>> >> not
>> >> > > > reproducible manually)
>> >> > > >
>> >> > > >    4.2) Old regressions on 1 platform
>> >> > > >         api.java.rmi.basicexception (ERROR in Linux x86,
>> >> HARMONY-5823)
>> >> > > >         api.java.rmi.basicregistry.RemoteServerTest (ERROR
in
>> Linux
>> >> x86,
>> >> > > > HARMONY-5823)
>> >> > > >         jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows
x86_64,
>> in
>> >> M6
>> >> > > it
>> >> > > > is recorded as regression since M4)
>> >> > > >         java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR
on
>> Linux
>> >> x86,
>> >> > > > recorded as Timeout, not reproducible
>> >> > > >  in M6)
>> >> > > >         reg.vm.btest5717 (ERROR on Windows X86_64, recorded
as
>> >> "timeout,
>> >> > > > the test is too heavy" in M6)
>> >> > > >         jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1
>> (FAILED
>> >> in
>> >> > > > windows x86_64, recorded as "looks like
>> >> > > > issue in test" in M6)
>> >> > > >         jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on
windows
>> >> x86_64,
>> >> > > in
>> >> > > > M6 it is recorded as not regression and started to fail on
M5)
>> >> > > >         reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64,
>> recorded
>> >> > > also
>> >> > > > failed on M3 and might be issue with the test)
>> >> > > >
>> >> > > >    4.3) New regressions on 1 platform (looks intermittent)
>> >> > > >         reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
>> >> > > >         func.reg.jit.btest5710.Btest5710 (FAILED in Linux
x86)
>> >> > > >         api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06
>> >> (ERROR
>> >> > > > in Linux x86)
>> >> > > >         api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05
>> >> (ERROR
>> >> > > > in Linux x86)
>> >> > > >
>> >> > > >  5) JDKTools Tests:
>> >> > > >    Several timeouts are observed in Linux x86 snapshot running.
>> They
>> >> are:
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001
>> >> > > >
>> >> > > >
>> org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
>> >> > > >    The Linux-only timeouts are also observed in the integrity
>> testing
>> >> > > > results.
>> >> > > >    JIRA HARMONY-5833 has been filed for one of them.
>> >> > > >
>> >> > > >  6) JettyScenario:
>> >> > > >    The Linux x86 running failed because of the unresolved
>> >> HARMONY-5219.
>> >> > > >
>> >> > > >  7) Reliability:
>> >> > > >    Several failures are observed in windows x86_64 running.
>> >> > > >    7.1) Old regressions
>> >> > > >         api.net.DatagramTest (HARMONY-5531)
>> >> > > >         api.text.DecimalFormat_Locales - (in M6 it is recorded
as
>> >> also
>> >> > > > intermittent in M5)
>> >> > > >
>> >> > > >    7.2) New/intemittent regressions
>> >> > > >         api.kernel.thread.ThreadLocalTest.ThreadLocalTest
>> >> > > >         api.kernel.exec.RunExec
>> >> > > >
>> >> > > >  8) Stress
>> >> > > >    Different test cases failed on different platforms.
>> >> > > >    8.1) Timeouts on Linux x86.
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011
>> >> > > >
>> >> > > >    8.2) Failed cases on Windows x86_64 with unknown reasons.
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses
>> >> > > >
>> >> > > >    I have not found records for them.
>> >> > > >
>> >> > > >  9) Strut_test
>> >> > > >    Broken with same error report as M6.
>> >> > > >
>> >> > > >  10) Eclipse Hello World Application.
>> >> > > >    Although the testing framework just reports EUT-API status
as
>> >> > > "PASSED".
>> >> > > > A fresh workspace running just failed in configuration stage
(
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e
>> >> > > > )
>> >> > > > since SVN commit 641928 (which committed patch for HARMONY-4569).
>> >> > > >
>> >> > > > thanks,
>> >> > > > chunrong
>> >> > > >
>> >> > >
>> >>
>> >
>>
>

Mime
View raw message