harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Beliaev" <vladimir.k.beli...@gmail.com>
Subject Re: [testing] M4 candidate (r603534) testing status.
Date Fri, 14 Dec 2007 13:40:38 GMT
Updated status:

>     - Linux_x86: 96.72%
>       1257 issues in
org.eclipse.jdt.core.tests.compiler.regression.TestAll - they are
intermittent, must be ignored.

I did not reproduced them locally.

>  1 issue in org.eclipse.jdt.text.tests.JdtTextTestSuite - I'm checking it
now, it looks like intermittent failure

Also not reproducible.

>     - Linux_x86_64: pass rate 95.29%
>     Fixing the crashes must allow us to get 98% pass rate, the stand along
failures/errors are EFL candidates. So:
>     ...
>     Crashed org.eclipse.team.tests.ccvs.core.AllTests - new for me, I'm
checking this locally.

Not reproduced locally.

So remained issues for resolution are:

HARMONY-5306 [buildtest][EUT] swt suite intermittently crashes by timeout
(Linux x86 regressions)
-----
HARMONY-5303 [drlvm][EUT] regression in coreresources on Linux x86_64 (Linux
x86_64 98%)
HARMONY-5305 [drlvm][EUT] regression in jdtdebug in Linux x86_64 (Linux
x86_64 98%)
HARMONY-5309 [EUT] 274 NPE in checkAndTagAsMalformed on Linux x86_64 (Linux
x86_64 98%)

Thanks
Vladimir

2007/12/14, Vladimir Beliaev <vladimir.k.beliaev@gmail.com>:
>
> Hello, Stepan, Tim,
>
> > * Eclipse Unit Tests 3.3
> >    - Linux_x86: 4 failures
> >    - Windows_x86: pass rate 96.92%
> >    - Linux_x86_64: pass rate 95.29%
>
> Actually according to [1] the status is:
> ==========
>     - Windows_x86: pass rate 99.77% (but not 96.92%). There are 3 expected
> failures which are in EFL already & which caused the winx86 status is RED.
> ==========
>     - Linux_x86: 96.72%
>       1257 issues in
> org.eclipse.jdt.core.tests.compiler.regression.TestAll - they are
> intermittent, must be ignored.
>       1 issue in org.eclipse.jdt.text.tests.JdtTextTestSuite - I'm
> checking it now, it looks like intermittent failure
>       6 issues in org.eclipse.jdt.ui.tests.AutomatedSuite - HARMONY-5307,
> looks like invalid tests - I'm checking this now.
> ==========
>     - Linux_x86_64: pass rate 95.29%
>
>       Fixing the crashes must allow us to get 98% pass rate, the stand
> along failures/errors are EFL candidates. So:
>
>       Crashed org.eclipse.core.tests.resources.AutomatedTests - real
> crash, HARMONY-5303, the JIRA is being evaluated now.
>
>       Crashed org.eclipse.jdt.debug.tests.AutomatedSuite - real crash,
> HARMONY-5305, the JIRA is being evaluated now.
>
>       Crashed org.eclipse.team.tests.ccvs.core.AllTests - new for me, I'm
> checking this locally.
>
> Thanks
> Vladimir
>
> [1]
> http://people.apache.org/~mloenko/snapshot_testing/script/r603534/index.html
>
>
> 2007/12/14, Stepan Mishura <stepan.mishura@gmail.com>:
> >
> > Hi,
> >
> > Please see testing status[1] for r603534 below.
> > Note to my comments:
> > - I marked tests as regression if they passed on M3 [2]
> > - I question tests as intermittent if they passed on the previous
> > snapshot (r603433) [3]
> > - There are some tests that failed on M3 too. (move them to the exclude
> > lists?)
> > - Not all failures have JIRA issue assigned so please help me to find
> > related JIRA number (if there is any).
> >
> > All comments and clarifications are greatly appreciated!
> >
> > =====================================================
> > We have the next suites failed on the snapshot.
> >
> > * Eclipse Unit Tests 3.3
> >   - Linux_x86: 4 failures
> >   - Windows_x86: pass rate 96.92%
> >   - Linux_x86_64: pass rate 95.29%
> >
> > Would be nice to get failures evaluation.
> > We definitely improved stability of the Eclipse Unit Test suite since
> > M3 (Thanks to Vladimir Beliaev).
> > But the are still some issues such as intermittent suite crashes (that
> > make me very upset).
> >
> > * EGAx48:
> >   - Linux_x86 & Linux_x86_64 : failed. (HARMONY-5250 / regression?)
> >   - Windows_x86:
> >      Still no status – the scenario causes CC host down after running
> > several hours.
> >      Currently I can not figure out if this is the host's issue or
> > problem introduced by Harmony.
> >
> >
> > * EHWA(GUI)
> >   - Linux_x86 & Linux_x86_64 : failed.
> >   The scenario is quite unstable. Most probably I'll drop the
> > scenario in M5 from snapshot test and use only EHWA-API.
> >
> >
> > * Functional:
> >      F_KeyFactoryTest_01: regression on all platforms (HARMONY-4857)
> >
> > - Linux_x86:
> >      JComponent: intermittent?
> > - Linux_x86_64:
> >      btest6353: fails on M3 too
> > - Windows_x86_64:
> >      DataOutputStream, writeObjectReadObject0004, IfElse1, btest5717,
> > btest7214 : intermittent?
> >      btest6353: fails on M3 too
> >
> > * GUT:
> >    ContainerTest, JAASSecurityTest - fail on all platforms
> >      HARMONY-4895 says that this bug in Geronimo and not a regression.
> >
> > - Linux_x86:
> >    HARMONY-5302 - says that this a bug in tests (I don't understand
> > then why they passed on M3)
> >    security\jaas\ConfigurationEntryTest
> >    security\jaas\LoginKerberousNonGeronimoTest
> >    security\jaas\LoginKerberousTest
> >    security\jaas\LoginSQLTest
> >    security\network\protocol\SubjectCarryingProtocolTest
> >    security\remoting\jmx\RemoteLoginTest
> >
> > - Linux_x86_64:   53 failures (11 on M3)
> > - Windows_x86_64: 22 failures (28 on M3)
> >
> >
> > * JDKTools
> > - Windows_x86_64: 3 failures by TimeoutException.
> > Regression? I previously incorrectly reported that they are
> > intermittent. They stably fail on M4 builds.
> >
> > * Reliability
> >
> >    Failed on x86 platforms but they also failed on M3
> >    api.net.HttpConnectionTest (I wonder why HARMONY-5266 didn't fixed
> > it)
> >    api.net.SingleConnectTest
> >
> > - Linux_x86:
> >
> >    Regresions:
> >    api.kernel.object.ObjectFinalizeTest
> >    api.kernel.throwable.StackTraceExcptsTest
> >    api.text.DecimalFormat_Locales
> >    api.zip.ZipEntryIterateThreadingTest
> >    api.zip.ZlibTest
> >    vm.finalization.FinalizeThrowRestoreTest
> >    vm.stack.StackUnwindTest
> >
> >    They also failed on M3:
> >    api.nio.channels.filechannel.FileChannelMapTest
> >    api.nio.channels.filechannel.MappedByteBufferForceTest
> >    api.serialization.SerializableClassesTest
> >
> > - Windows_x86
> >
> >    Regresions:
> >    api.kernel.string.InternThreadingTest - regression
> >    api.kernel.thread.Calculation.CalcTest
> >    api.kernel.thread.RecursiveThreadTest
> >    api.kernel.thread.StackTraceTest
> >    api.kernel.thread.Synchronization
> >    api.kernel.thread.ThreadArrayTest
> >    api.kernel.threadgroup.EnumerateTest
> >    api.net.DatagramTest
> >    api.serialization.SerializableClassesTest
> >    vm.stack.StackUnwindTest
> >
> > - Linux_x86_64:
> >
> >    Regressions:
> >    vm.classloading.ClassCastTest
> >
> > * Stress
> >
> > - Linux_x86:
> >    StackUnwindingManyObjectsTests: 5 fails on M3 too (HARMONY-5158 /
> > test issue?)
> >    Fragmentation, FragmentationFinalizer, FragmentationReference:
> > HARMONY-5159 (wasn't run during M3 so not regression)
> >
> > - Windows_x86:
> >     StackUnwindingManyObjectsTests: 5 fails on M3 too (HARMONY-5158 /
> > test issue?)
> >     FragmentationReference: HARMONY-5159 (wasn't run during M3 so not
> > regression)
> >
> > - Linux_x86_64:
> >    jni.arguments,jni.arrays - fails on M3 too so not regression /
> > HARMONY-????
> >    MEMORY001, MEMORY003 - HARMONY-5047 (fails on M3 too so not
> > regression)
> >
> > - Windows_x86_64:
> >    NotSynchThreads.ManyClasses - crash (HARMONY-5003), regression
> >    Fragmentation, FragmentationFinalizer, FragmentationReference:
> > HARMONY-5159 (wasn't run during M3 so not regression)
> >    gc.mem.MemoryTest3 - regression / HARMONY-????
> >    jni.stack - fails on M3 too so not regression / HARMONY-????
> >    EVENT014- crash (HARMONY-????)/ regression
> >    MEMORY001, MEMORY003 - HARMONY-5047 (fails on M3 too so not
> > regression)
> >    StressThreads12Test - crash (HARMONY-????)/ regression
> >
> > * VTS VM
> > - Windows_x86:
> >      dstore_20701 - intermittent?
> >
> > [1]
> > http://people.apache.org/~mloenko/snapshot_testing/script/r603534/index.html
> > [2]
> > http://people.apache.org/~mloenko/snapshot_testing/script/r580985/index.html
> > [3]
> > http://people.apache.org/~mloenko/snapshot_testing/script/r603433/index.html
> >
> > Thanks,
> > Stepan Mishura
> > Intel Enterprise Solutions Software Division
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message