harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Beliaev" <vladimir.k.beli...@gmail.com>
Subject Re: [general][M3] release candidate status
Date Fri, 28 Sep 2007 09:11:46 GMT
Helllo, Stepan,

I confirm it is high probability that there is nothing to be fixed in
Harmony for Eclipse Unit Tests now to significantly change the status. I
mean the M3 snapshot is pretty "well done" in respect to EUT.

*Still the EUT running script still may be improved*.

For example I can redo them to run each EUT Suite separately making sure all
resources are cleaned up after each suite run (still this won't cleanup $TMP
or $HOME directory because I do not have the exact list of file names
created by EUT). Still the current issues wwith *jdtcorecompiler* better be
resolved in Eclipse (but not workarounded in Harmony), so I'm not sure it is
a good solution.

Or I can incorporate to these scripts the creation, specifying and
cleanup-ing $HOME and $TMP directories.

So, please write if you have proposals for on EUT running scripts.

Vladimir Beliaev

2007/9/28, Stepan Mishura <stepan.mishura@gmail.com>:

> Hi all,
> OK. Here is testing status for r579330 M3 milestone candidate.
> It is based on testing status page[1] and investigation reports for
> suites on dev-list.
> It is provided to decide whether r579330 is OK to be released or we
> should fix found issues and build next release candidate.
> Status reporter please correct me if I miss something.
> I'd like to say that this in this milestone we used BTI for the first
> time to get 100% of testing results. Some issues in BTI were fixed but
> there are some still to be resolved, for example instability of
> several suites, mostly Eclipse unit tests on Linux.
> The following suites passed on Linux/Windows ia32 platform: Ant
> Scenario (or self-hosting), Axis application, Dacapo, DRLVM tests,
> DRLVM regression tests, Jetty scenario, Scimark, Struts scenario,
> Tomcat scenario
> The the next list of issues we have (per suite):
> - Classlibrary unit tests:
>    1 crash on Windows that is hard to reproduce:
>    1 crash on Linux that is reproducible and it is bug in DRLVM
> (HARMONY-????):
>    1 test failure on Linux (incorrect test HARMONY-????)
> - Eclipse unit tests 3.2
>    Windows: 4 failures. 1 failure is expected (exclude list should be
> updated). 3 failure are not reproducible
>    Linux: crash of
> org.eclipse.jdt.core.tests.compiler.regression.TestAll. It crashes due
> timeout
> - Eclipse unit tests 3.3
>   Windows: 49 failures, pass rate 99.77%. We haven't tried the suite
> on M2 so it is a good baseline for M3.
>   Linux: the same problem as for 3.2. I've setup another CC Linux
> host for the suite and see no crashes, pass rate is 99.45%
> So I'm going to publish these reports for M3 and run 3.2 suite on
> these lucky host to obtain results without the crash.
> - Eclipse Geronimo Application (EGA) scenario x 48h
>   Windows: there may be and issue with the scenario automation - the
> scenario run more then 8 hours. As I understand several iterations
> were done successfully. And it failed because of missing window, there
> are no crash/exceptions in the log.
> Also the scenario passed on Linux.
> - Eclipse TPTP Tests
> 1 failure on Linux - most probably intermittent
> - Functional Test Suite
>   21 failures on Windows most of then failure due to timeout. 1
> regression (HARMONY-4857)
>   Uploading Linux failed - I'm going to check why. From the first
> glance the results are same as for Windows.
> - Geronimo Unit Tests
> 2 possible(I haven't seen analysis report) regressions on
> Windows/Linux. These test passed on M2:
> org.apache.geronimo.tomcat.ContainerTest
> org.apache.geronimo.tomcat.JAASSecurityTest
> - JDKTools Tests
> 1 failure on Windows. The test stably fails when it runs in a bunch
> with others and may fail if run standalone :
> - Reliability Test Suite
> The status is a bit unclear to me. There is 1 test is incorrect
> test. And some tests failed because of timeout but its increase didn't
> help.
> Also some tests were removed from exclude list for r579330 but some
> of them failed.
> - Stress Test Suite
> 3 failures on Windows and 11 on Linux - I haven't seen analysis report
> - VTS VM Test Suite
> 1 failure on Linux - the test passed on previous snapshot should be
> analysed.
> Any comments are welcome.
> [1]
> http://people.apache.org/~mloenko/snapshot_testing/script/r579330/index.html
> Thanks,
> Stepan Mishura
> Intel Enterprise Solutions Software Division

Vladimir Beliaev
Intel Middleware Products Division

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message