harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stepan Mishura" <stepan.mish...@gmail.com>
Subject [testing] M4 candidate (r603534) testing status.
Date Fri, 14 Dec 2007 08:09:08 GMT
Hi,

Please see testing status[1] for r603534 below.
Note to my comments:
- I marked tests as regression if they passed on M3 [2]
- I question tests as intermittent if they passed on the previous
snapshot (r603433) [3]
- There are some tests that failed on M3 too. (move them to the exclude lists?)
- Not all failures have JIRA issue assigned so please help me to find
related JIRA number (if there is any).

All comments and clarifications are greatly appreciated!

=====================================================
We have the next suites failed on the snapshot.

* Eclipse Unit Tests 3.3
   - Linux_x86: 4 failures
   - Windows_x86: pass rate 96.92%
   - Linux_x86_64: pass rate 95.29%

Would be nice to get failures evaluation.
We definitely improved stability of the Eclipse Unit Test suite since
M3 (Thanks to Vladimir Beliaev).
But the are still some issues such as intermittent suite crashes (that
make me very upset).

* EGAx48:
   - Linux_x86 & Linux_x86_64 : failed. (HARMONY-5250 / regression?)
   - Windows_x86:
      Still no status – the scenario causes CC host down after running
several hours.
      Currently I can not figure out if this is the host's issue or
problem introduced by Harmony.


* EHWA(GUI)
   - Linux_x86 & Linux_x86_64 : failed.
   The scenario is quite unstable. Most probably I'll drop the
scenario in M5 from snapshot test and use only EHWA-API.


* Functional:
      F_KeyFactoryTest_01: regression on all platforms (HARMONY-4857)

  - Linux_x86:
      JComponent: intermittent?
  - Linux_x86_64:
      btest6353: fails on M3 too
  - Windows_x86_64:
      DataOutputStream, writeObjectReadObject0004, IfElse1, btest5717,
btest7214 : intermittent?
      btest6353: fails on M3 too

* GUT:
    ContainerTest, JAASSecurityTest - fail on all platforms
      HARMONY-4895 says that this bug in Geronimo and not a regression.

  - Linux_x86:
    HARMONY-5302 - says that this a bug in tests (I don't understand
then why they passed on M3)
    security\jaas\ConfigurationEntryTest
    security\jaas\LoginKerberousNonGeronimoTest
    security\jaas\LoginKerberousTest
    security\jaas\LoginSQLTest
    security\network\protocol\SubjectCarryingProtocolTest
    security\remoting\jmx\RemoteLoginTest

  - Linux_x86_64:   53 failures (11 on M3)
  - Windows_x86_64: 22 failures (28 on M3)


* JDKTools
  - Windows_x86_64: 3 failures by TimeoutException.
  Regression? I previously incorrectly reported that they are
intermittent. They stably fail on M4 builds.

* Reliability

    Failed on x86 platforms but they also failed on M3
    api.net.HttpConnectionTest (I wonder why HARMONY-5266 didn't fixed it)
    api.net.SingleConnectTest

  - Linux_x86:

    Regresions:
    api.kernel.object.ObjectFinalizeTest
    api.kernel.throwable.StackTraceExcptsTest
    api.text.DecimalFormat_Locales
    api.zip.ZipEntryIterateThreadingTest
    api.zip.ZlibTest
    vm.finalization.FinalizeThrowRestoreTest
    vm.stack.StackUnwindTest

    They also failed on M3:
    api.nio.channels.filechannel.FileChannelMapTest
    api.nio.channels.filechannel.MappedByteBufferForceTest
    api.serialization.SerializableClassesTest

  - Windows_x86

    Regresions:
    api.kernel.string.InternThreadingTest - regression
    api.kernel.thread.Calculation.CalcTest
    api.kernel.thread.RecursiveThreadTest
    api.kernel.thread.StackTraceTest
    api.kernel.thread.Synchronization
    api.kernel.thread.ThreadArrayTest
    api.kernel.threadgroup.EnumerateTest
    api.net.DatagramTest
    api.serialization.SerializableClassesTest
    vm.stack.StackUnwindTest

  - Linux_x86_64:

    Regressions:
    vm.classloading.ClassCastTest

* Stress

  - Linux_x86:
    StackUnwindingManyObjectsTests: 5 fails on M3 too (HARMONY-5158 /
test issue?)
    Fragmentation, FragmentationFinalizer, FragmentationReference:
HARMONY-5159 (wasn't run during M3 so not regression)

  - Windows_x86:
     StackUnwindingManyObjectsTests: 5 fails on M3 too (HARMONY-5158 /
test issue?)
     FragmentationReference: HARMONY-5159 (wasn't run during M3 so not
regression)

  - Linux_x86_64:
    jni.arguments,jni.arrays - fails on M3 too so not regression / HARMONY-????
    MEMORY001, MEMORY003 - HARMONY-5047 (fails on M3 too so not regression)

  - Windows_x86_64:
    NotSynchThreads.ManyClasses - crash (HARMONY-5003), regression
    Fragmentation, FragmentationFinalizer, FragmentationReference:
HARMONY-5159 (wasn't run during M3 so not regression)
    gc.mem.MemoryTest3 - regression / HARMONY-????
    jni.stack - fails on M3 too so not regression / HARMONY-????
    EVENT014- crash (HARMONY-????)/ regression
    MEMORY001, MEMORY003 - HARMONY-5047 (fails on M3 too so not regression)
    StressThreads12Test - crash (HARMONY-????)/ regression

* VTS VM
  - Windows_x86:
      dstore_20701 - intermittent?

[1] http://people.apache.org/~mloenko/snapshot_testing/script/r603534/index.html
[2] http://people.apache.org/~mloenko/snapshot_testing/script/r580985/index.html
[3] http://people.apache.org/~mloenko/snapshot_testing/script/r603433/index.html

Thanks,
Stepan Mishura
Intel Enterprise Solutions Software Division

Mime
View raw message