harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexei Fedotov" <alexei.fedo...@gmail.com>
Subject Re: [general][M3] release candidate status
Date Mon, 01 Oct 2007 18:17:53 GMT
Hello, Stepan,
> 3 failures on Windows and 11 on Linux - I haven't seen analysis report

I have investigated log files for all 14 failures, and all of them are
due to test errors. Most of them are due to timeouts, and others are
due to OutOfMemoryError. All of these parameters are hard-coded and
not automatically adjusted during execution, so changes in VM
performance produce such unexpected problems.

Even if some timeouts could be caused by Harmony implementation
deadlocks, the correct hing is to address the tests first since they
are much easier to fix.

BTW,
There are also massive failures on Win64 platform because TI is called
with JIT. If you have a good example of ant build.xml which checks for
64 bit platform (or even uses platfrom-dependent excludes), this would
be a great help.

BTW2,
Nearly all memory fragmentation tests created for H-4114 produce a
crash in the function gc_heap_slot_write_ref(). Two old tests crash in
trace_object() function. I will re-run the tests using debug build.
Strangely, these crashes do not apear in CC report.

Thanks.

On 9/28/07, Stepan Mishura <stepan.mishura@gmail.com> wrote:
> Hi all,
>
> OK. Here is testing status for r579330 M3 milestone candidate.
> It is based on testing status page[1] and investigation reports for
> suites on dev-list.
> It is provided to decide whether r579330 is OK to be released or we
> should fix found issues and build next release candidate.
>
> Status reporter please correct me if I miss something.
>
> I'd like to say that this in this milestone we used BTI for the first
> time to get 100% of testing results. Some issues in BTI were fixed but
> there are some still to be resolved, for example instability of
> several suites, mostly Eclipse unit tests on Linux.
>
>
> The following suites passed on Linux/Windows ia32 platform: Ant
> Scenario (or self-hosting), Axis application, Dacapo, DRLVM tests,
> DRLVM regression tests, Jetty scenario, Scimark, Struts scenario,
> Tomcat scenario
>
> The the next list of issues we have (per suite):
> - Classlibrary unit tests:
>    1 crash on Windows that is hard to reproduce:
>    1 crash on Linux that is reproducible and it is bug in DRLVM (HARMONY-????):
>    1 test failure on Linux (incorrect test HARMONY-????)
>
> - Eclipse unit tests 3.2
>    Windows: 4 failures. 1 failure is expected (exclude list should be
> updated). 3 failure are not reproducible
>    Linux: crash of
> org.eclipse.jdt.core.tests.compiler.regression.TestAll. It crashes due
> timeout
>
> - Eclipse unit tests 3.3
>   Windows: 49 failures, pass rate 99.77%. We haven't tried the suite
> on M2 so it is a good baseline for M3.
>   Linux: the same problem as for 3.2. I've setup another CC Linux
> host for the suite and see no crashes, pass rate is 99.45%
>
>  So I'm going to publish these reports for M3 and run 3.2 suite on
> these lucky host to obtain results without the crash.
>
> - Eclipse Geronimo Application (EGA) scenario x 48h
>   Windows: there may be and issue with the scenario automation - the
> scenario run more then 8 hours. As I understand several iterations
> were done successfully. And it failed because of missing window, there
> are no crash/exceptions in the log.
>
>  Also the scenario passed on Linux.
>
> - Eclipse TPTP Tests
>  1 failure on Linux - most probably intermittent
>
> - Functional Test Suite
>   21 failures on Windows most of then failure due to timeout. 1
> regression (HARMONY-4857)
>   Uploading Linux failed - I'm going to check why. From the first
> glance the results are same as for Windows.
>
> - Geronimo Unit Tests
>  2 possible(I haven't seen analysis report) regressions on
> Windows/Linux. These test passed on M2:
>  org.apache.geronimo.tomcat.ContainerTest
>  org.apache.geronimo.tomcat.JAASSecurityTest
>
> - JDKTools Tests
>  1 failure on Windows. The test stably fails when it runs in a bunch
> with others and may fail if run standalone :
>
> - Reliability Test Suite
>  The status is a bit unclear to me. There is 1 test is incorrect
> test. And some tests failed because of timeout but its increase didn't
> help.
>  Also some tests were removed from exclude list for r579330 but some
> of them failed.
>
> - Stress Test Suite
>  3 failures on Windows and 11 on Linux - I haven't seen analysis report
>
> - VTS VM Test Suite
>  1 failure on Linux - the test passed on previous snapshot should be analysed.
>
> Any comments are welcome.
>
> [1] http://people.apache.org/~mloenko/snapshot_testing/script/r579330/index.html
>
> Thanks,
> Stepan Mishura
> Intel Enterprise Solutions Software Division
>


-- 
With best regards,
Alexei,
ESSD, Intel

Mime
View raw message