harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stepan Mishura" <stepan.mish...@gmail.com>
Subject [general][M3] release candidate status
Date Fri, 28 Sep 2007 08:02:33 GMT
Hi all,

OK. Here is testing status for r579330 M3 milestone candidate.
It is based on testing status page[1] and investigation reports for
suites on dev-list.
It is provided to decide whether r579330 is OK to be released or we
should fix found issues and build next release candidate.

Status reporter please correct me if I miss something.

I'd like to say that this in this milestone we used BTI for the first
time to get 100% of testing results. Some issues in BTI were fixed but
there are some still to be resolved, for example instability of
several suites, mostly Eclipse unit tests on Linux.


The following suites passed on Linux/Windows ia32 platform: Ant
Scenario (or self-hosting), Axis application, Dacapo, DRLVM tests,
DRLVM regression tests, Jetty scenario, Scimark, Struts scenario,
Tomcat scenario

The the next list of issues we have (per suite):
- Classlibrary unit tests:
    1 crash on Windows that is hard to reproduce:
    1 crash on Linux that is reproducible and it is bug in DRLVM (HARMONY-????):
    1 test failure on Linux (incorrect test HARMONY-????)

- Eclipse unit tests 3.2
    Windows: 4 failures. 1 failure is expected (exclude list should be
updated). 3 failure are not reproducible
    Linux: crash of
org.eclipse.jdt.core.tests.compiler.regression.TestAll. It crashes due
timeout

- Eclipse unit tests 3.3
   Windows: 49 failures, pass rate 99.77%. We haven't tried the suite
on M2 so it is a good baseline for M3.
   Linux: the same problem as for 3.2. I've setup another CC Linux
host for the suite and see no crashes, pass rate is 99.45%

  So I'm going to publish these reports for M3 and run 3.2 suite on
these lucky host to obtain results without the crash.

- Eclipse Geronimo Application (EGA) scenario x 48h
   Windows: there may be and issue with the scenario automation - the
scenario run more then 8 hours. As I understand several iterations
were done successfully. And it failed because of missing window, there
are no crash/exceptions in the log.

  Also the scenario passed on Linux.

- Eclipse TPTP Tests
  1 failure on Linux - most probably intermittent

- Functional Test Suite
   21 failures on Windows most of then failure due to timeout. 1
regression (HARMONY-4857)
   Uploading Linux failed - I'm going to check why. From the first
glance the results are same as for Windows.

- Geronimo Unit Tests
  2 possible(I haven't seen analysis report) regressions on
Windows/Linux. These test passed on M2:
  org.apache.geronimo.tomcat.ContainerTest
  org.apache.geronimo.tomcat.JAASSecurityTest

- JDKTools Tests
  1 failure on Windows. The test stably fails when it runs in a bunch
with others and may fail if run standalone :

- Reliability Test Suite
  The status is a bit unclear to me. There is 1 test is incorrect
test. And some tests failed because of timeout but its increase didn't
help.
  Also some tests were removed from exclude list for r579330 but some
of them failed.

- Stress Test Suite
  3 failures on Windows and 11 on Linux - I haven't seen analysis report

- VTS VM Test Suite
  1 failure on Linux - the test passed on previous snapshot should be analysed.

Any comments are welcome.

[1] http://people.apache.org/~mloenko/snapshot_testing/script/r579330/index.html

Thanks,
Stepan Mishura
Intel Enterprise Solutions Software Division

Mime
View raw message