harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Beliaev" <vladimir.k.beli...@gmail.com>
Subject Re: [testing] M4 candidate (r603534) testing status.
Date Fri, 14 Dec 2007 10:50:25 GMT
Hello, Stepan, Tim,

> * Eclipse Unit Tests 3.3
>    - Linux_x86: 4 failures
>    - Windows_x86: pass rate 96.92%
>    - Linux_x86_64: pass rate 95.29%

Actually according to [1] the status is:
==========
    - Windows_x86: pass rate 99.77% (but not 96.92%). There are 3 expected
failures which are in EFL already & which caused the winx86 status is RED.
==========
    - Linux_x86: 96.72%
      1257 issues in
org.eclipse.jdt.core.tests.compiler.regression.TestAll- they are
intermittent, must be ignored.
      1 issue in org.eclipse.jdt.text.tests.JdtTextTestSuite - I'm checking
it now, it looks like intermittent failure
      6 issues in org.eclipse.jdt.ui.tests.AutomatedSuite - HARMONY-5307,
looks like invalid tests - I'm checking this now.
==========
    - Linux_x86_64: pass rate 95.29%

      Fixing the crashes must allow us to get 98% pass rate, the stand along
failures/errors are EFL candidates. So:

      Crashed org.eclipse.core.tests.resources.AutomatedTests - real crash,
HARMONY-5303, the JIRA is being evaluated now.

      Crashed org.eclipse.jdt.debug.tests.AutomatedSuite - real crash,
HARMONY-5305, the JIRA is being evaluated now.

      Crashed org.eclipse.team.tests.ccvs.core.AllTests - new for me, I'm
checking this locally.

Thanks
Vladimir

[1]
http://people.apache.org/~mloenko/snapshot_testing/script/r603534/index.html


2007/12/14, Stepan Mishura <stepan.mishura@gmail.com>:
>
> Hi,
>
> Please see testing status[1] for r603534 below.
> Note to my comments:
> - I marked tests as regression if they passed on M3 [2]
> - I question tests as intermittent if they passed on the previous
> snapshot (r603433) [3]
> - There are some tests that failed on M3 too. (move them to the exclude
> lists?)
> - Not all failures have JIRA issue assigned so please help me to find
> related JIRA number (if there is any).
>
> All comments and clarifications are greatly appreciated!
>
> =====================================================
> We have the next suites failed on the snapshot.
>
> * Eclipse Unit Tests 3.3
>   - Linux_x86: 4 failures
>   - Windows_x86: pass rate 96.92%
>   - Linux_x86_64: pass rate 95.29%
>
> Would be nice to get failures evaluation.
> We definitely improved stability of the Eclipse Unit Test suite since
> M3 (Thanks to Vladimir Beliaev).
> But the are still some issues such as intermittent suite crashes (that
> make me very upset).
>
> * EGAx48:
>   - Linux_x86 & Linux_x86_64 : failed. (HARMONY-5250 / regression?)
>   - Windows_x86:
>      Still no status – the scenario causes CC host down after running
> several hours.
>      Currently I can not figure out if this is the host's issue or
> problem introduced by Harmony.
>
>
> * EHWA(GUI)
>   - Linux_x86 & Linux_x86_64 : failed.
>   The scenario is quite unstable. Most probably I'll drop the
> scenario in M5 from snapshot test and use only EHWA-API.
>
>
> * Functional:
>      F_KeyFactoryTest_01: regression on all platforms (HARMONY-4857)
>
> - Linux_x86:
>      JComponent: intermittent?
> - Linux_x86_64:
>      btest6353: fails on M3 too
> - Windows_x86_64:
>      DataOutputStream, writeObjectReadObject0004, IfElse1, btest5717,
> btest7214 : intermittent?
>      btest6353: fails on M3 too
>
> * GUT:
>    ContainerTest, JAASSecurityTest - fail on all platforms
>      HARMONY-4895 says that this bug in Geronimo and not a regression.
>
> - Linux_x86:
>    HARMONY-5302 - says that this a bug in tests (I don't understand
> then why they passed on M3)
>    security\jaas\ConfigurationEntryTest
>    security\jaas\LoginKerberousNonGeronimoTest
>    security\jaas\LoginKerberousTest
>    security\jaas\LoginSQLTest
>    security\network\protocol\SubjectCarryingProtocolTest
>    security\remoting\jmx\RemoteLoginTest
>
> - Linux_x86_64:   53 failures (11 on M3)
> - Windows_x86_64: 22 failures (28 on M3)
>
>
> * JDKTools
> - Windows_x86_64: 3 failures by TimeoutException.
> Regression? I previously incorrectly reported that they are
> intermittent. They stably fail on M4 builds.
>
> * Reliability
>
>    Failed on x86 platforms but they also failed on M3
>    api.net.HttpConnectionTest (I wonder why HARMONY-5266 didn't fixed it)
>    api.net.SingleConnectTest
>
> - Linux_x86:
>
>    Regresions:
>    api.kernel.object.ObjectFinalizeTest
>    api.kernel.throwable.StackTraceExcptsTest
>    api.text.DecimalFormat_Locales
>    api.zip.ZipEntryIterateThreadingTest
>    api.zip.ZlibTest
>    vm.finalization.FinalizeThrowRestoreTest
>    vm.stack.StackUnwindTest
>
>    They also failed on M3:
>    api.nio.channels.filechannel.FileChannelMapTest
>    api.nio.channels.filechannel.MappedByteBufferForceTest
>    api.serialization.SerializableClassesTest
>
> - Windows_x86
>
>    Regresions:
>    api.kernel.string.InternThreadingTest - regression
>    api.kernel.thread.Calculation.CalcTest
>    api.kernel.thread.RecursiveThreadTest
>    api.kernel.thread.StackTraceTest
>    api.kernel.thread.Synchronization
>    api.kernel.thread.ThreadArrayTest
>    api.kernel.threadgroup.EnumerateTest
>    api.net.DatagramTest
>    api.serialization.SerializableClassesTest
>    vm.stack.StackUnwindTest
>
> - Linux_x86_64:
>
>    Regressions:
>    vm.classloading.ClassCastTest
>
> * Stress
>
> - Linux_x86:
>    StackUnwindingManyObjectsTests: 5 fails on M3 too (HARMONY-5158 /
> test issue?)
>    Fragmentation, FragmentationFinalizer, FragmentationReference:
> HARMONY-5159 (wasn't run during M3 so not regression)
>
> - Windows_x86:
>     StackUnwindingManyObjectsTests: 5 fails on M3 too (HARMONY-5158 /
> test issue?)
>     FragmentationReference: HARMONY-5159 (wasn't run during M3 so not
> regression)
>
> - Linux_x86_64:
>    jni.arguments,jni.arrays - fails on M3 too so not regression /
> HARMONY-????
>    MEMORY001, MEMORY003 - HARMONY-5047 (fails on M3 too so not regression)
>
> - Windows_x86_64:
>    NotSynchThreads.ManyClasses - crash (HARMONY-5003), regression
>    Fragmentation, FragmentationFinalizer, FragmentationReference:
> HARMONY-5159 (wasn't run during M3 so not regression)
>    gc.mem.MemoryTest3 - regression / HARMONY-????
>    jni.stack - fails on M3 too so not regression / HARMONY-????
>    EVENT014- crash (HARMONY-????)/ regression
>    MEMORY001, MEMORY003 - HARMONY-5047 (fails on M3 too so not regression)
>    StressThreads12Test - crash (HARMONY-????)/ regression
>
> * VTS VM
> - Windows_x86:
>      dstore_20701 - intermittent?
>
> [1]
> http://people.apache.org/~mloenko/snapshot_testing/script/r603534/index.html
> [2]
> http://people.apache.org/~mloenko/snapshot_testing/script/r580985/index.html
> [3]
> http://people.apache.org/~mloenko/snapshot_testing/script/r603433/index.html
>
> Thanks,
> Stepan Mishura
> Intel Enterprise Solutions Software Division
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message