harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexey Varlamov" <alexey.v.varla...@gmail.com>
Subject Re: [general][M3] release candidate status
Date Wed, 03 Oct 2007 06:27:16 GMT
2007/10/3, Stepan Mishura <stepan.mishura@gmail.com>:
> On 10/2/07, Tim Ellison <t.p.ellison@gmail.com> wrote:
> > Stepan Mishura wrote:
> > > On 10/2/07, Mikhail Loenko <mloenko@gmail.com> wrote:
> > >> Do we see any M3 blockers so far?
> > >
> > > As I understand there are no critical/blocker issues for r579330.
> >
> > ? there were a number of regression on that revision that were blockers
> > and have been fixed.
> >
>
> All fixes for issues raised on dev-list were included to the next
> milestone candidate. Did I miss any?
I'm not aware of any missed must-have patches, too.

>
> > > Roughly,
> > > - several regressions were found (most of them were fixed and included
> > > into the new milestone candidate r580985), BWT, I encourage everyone
> > > to use M3 umbrella for tracking [1]
> >
> > Agreed.  I see two still open against that umbrella task.  I think we
> > can unlink HARMONY-4865 (tptp failures), but HARMONY-4184 looks like a
> > candidate to fix, right?
> >
>
> Alexey (Varlamov) did a quick fix for HARMONY-4184.
>
> Alexey, could you update its status? Did your quick fix(r580932)
> resolve the issue?
Surely it did, I think it is enough for M3. Complete fix better be
done in M4, there is a bunch of issues around security policy which
are risky to fix in last minute.
>
> > > - there are couple of suites to be improved (too many timeouts and
> > > intermittent failures): Reliability/Stress/Functional
> > > - automation for some suites should to be fixed: EGA/EUT
> > > - few reports are missed: Geronimo Unit Tests (looks like regressions)
> >
> > This is all good stuff to capture in our list of things to do for M4.
> >
>
> Yes, I hope to fix all these issues for M4 to make auto-testing more reliable.
>
> Thanks,
> Stepan.
>
> > Hopefully we can declare M3 soonest and unfreeze the code.
> >
> > Regards,
> > Tim
> >
> > > [1] https://issues.apache.org/jira/browse/HARMONY-4843
> > >
> > > Thanks,
> > > Stepan.
> > >
> > >> Thanks,
> > >> Mikhail
> > >>
> > >> 2007/10/2, Stepan Mishura <stepan.mishura@gmail.com>:
> > >>> Hi,
> > >>>
> > >>> Currently, the next milestone candidate (r580985) is under testing.
> > >>>
> > >>> There were a number of investigation reports for r579330 and I'd like
> > >>> to update its summary to track the progress. All
> > >>>
> > >>> * Class library:
> > >>>    - Windows: 1 crash (hard to reproduce)
> > >>>    - Linux: 1 crash (HARMONY-4873), 1 failure (incorrect test - HARMONY-4872)
> > >>> * Eclipse 3.2 unit tests:
> > >>>    - Windows:
> > >>>        4 failures - 1 is expected, 3 hard to reproduce (exclude list
> > >>> was updated)
> > >>>    - Linux:
> > >>>        issue with org.eclipse.jdt.core.tests.compiler.regression.TestAll
suite
> > >>>        timeout still remains.
> > >>> * Eclipse 3.3 unit tests:
> > >>>    Success rate on Windows - 99.77% and Linux 99.46%.
> > >>>    This results will be taken as baseline for tracking regressions.
> > >>> * Eclipse Geronimo Application (EGA) scenario x 48h
> > >>>    - Windows:
> > >>>       failed after 8 running hours, most probably the problem
> > >>>      with scenario automation - no crashes/exceptions in the output
> > >>> * Eclipse TPTP Tests:
> > >>>       - Linux: 1 intermittent failure
> > >>> * Functional Test Suite:
> > >>>    4 regressions were found - HARMONY-4866, HARMONY-4874,
> > >>> HARMONY-4857, HARMONY-4861
> > >>>    6 tests are invalid - HARMONY-4864(2 cases), HARMONY-4859 (1
> > >>> case), JIRA for others??
> > >>>    HARMONY-4867 - should be defined whether this is regression or not
> > >>>    Also there are many failures due to timeout or intermittent failures.
> > >>>   The suite definitely must be improved.
> > >>> * Geronimo Unit Tests
> > >>>    2 regressions on Windows/Linux - no evaluation yet
> > >>> * JDKTools tests:
> > >>>    Windows: 1 failure - reproducible only if the whole suite is run.
> > >>> (HARMONY-4868)
> > >>> * Reliability Test Suite:
> > >>>     - Windows: 13 failures - according to the evaluation report most
> > >>> of them are intermittent and additional investigation is required
> > >>>    - Linux: 14 failures - the same as for Windows
> > >>> * Stress Test Suite:
> > >>>    - Windows: 3 failures
> > >>>    - Linux: 11 failures
> > >>>       Most of test failures due to timeout or OutOfMemoryError - the
> > >>> suite must be improved. There is HARMONY-4114 with 1Mb patch for the
> > >>> suite and harness but it is too risky to apply it during code freeze
-
> > >>> will be applied in M4.
> > >>> * VTS VM Test Suite
> > >>>    - Linux: 1 intermittent failure - problems in DRLVM shutdown code
> > >>> (HARMONY-????).
> > >>>
> > >>> Thanks,
> > >>> Stepan.
> > >>>
> > >>> On 9/28/07, Stepan Mishura <stepan.mishura@gmail.com> wrote:
> > >>>> Hi all,
> > >>>>
> > >>>> OK. Here is testing status for r579330 M3 milestone candidate.
> > >>>> It is based on testing status page[1] and investigation reports
for
> > >>>> suites on dev-list.
> > >>>> It is provided to decide whether r579330 is OK to be released or
we
> > >>>> should fix found issues and build next release candidate.
> > >>>>
> > >>>> Status reporter please correct me if I miss something.
> > >>>>
> > >>>> I'd like to say that this in this milestone we used BTI for the
first
> > >>>> time to get 100% of testing results. Some issues in BTI were fixed
but
> > >>>> there are some still to be resolved, for example instability of
> > >>>> several suites, mostly Eclipse unit tests on Linux.
> > >>>>
> > >>>>
> > >>>> The following suites passed on Linux/Windows ia32 platform: Ant
> > >>>> Scenario (or self-hosting), Axis application, Dacapo, DRLVM tests,
> > >>>> DRLVM regression tests, Jetty scenario, Scimark, Struts scenario,
> > >>>> Tomcat scenario
> > >>>>
> > >>>> The the next list of issues we have (per suite):
> > >>>> - Classlibrary unit tests:
> > >>>>    1 crash on Windows that is hard to reproduce:
> > >>>>    1 crash on Linux that is reproducible and it is bug in DRLVM
(HARMONY-????):
> > >>>>    1 test failure on Linux (incorrect test HARMONY-????)
> > >>>>
> > >>>> - Eclipse unit tests 3.2
> > >>>>    Windows: 4 failures. 1 failure is expected (exclude list should
be
> > >>>> updated). 3 failure are not reproducible
> > >>>>    Linux: crash of
> > >>>> org.eclipse.jdt.core.tests.compiler.regression.TestAll. It crashes
due
> > >>>> timeout
> > >>>>
> > >>>> - Eclipse unit tests 3.3
> > >>>>   Windows: 49 failures, pass rate 99.77%. We haven't tried the
suite
> > >>>> on M2 so it is a good baseline for M3.
> > >>>>   Linux: the same problem as for 3.2. I've setup another CC Linux
> > >>>> host for the suite and see no crashes, pass rate is 99.45%
> > >>>>
> > >>>>  So I'm going to publish these reports for M3 and run 3.2 suite
on
> > >>>> these lucky host to obtain results without the crash.
> > >>>>
> > >>>> - Eclipse Geronimo Application (EGA) scenario x 48h
> > >>>>   Windows: there may be and issue with the scenario automation
- the
> > >>>> scenario run more then 8 hours. As I understand several iterations
> > >>>> were done successfully. And it failed because of missing window,
there
> > >>>> are no crash/exceptions in the log.
> > >>>>
> > >>>>  Also the scenario passed on Linux.
> > >>>>
> > >>>> - Eclipse TPTP Tests
> > >>>>  1 failure on Linux - most probably intermittent
> > >>>>
> > >>>> - Functional Test Suite
> > >>>>   21 failures on Windows most of then failure due to timeout. 1
> > >>>> regression (HARMONY-4857)
> > >>>>   Uploading Linux failed - I'm going to check why. From the first
> > >>>> glance the results are same as for Windows.
> > >>>>
> > >>>> - Geronimo Unit Tests
> > >>>>  2 possible(I haven't seen analysis report) regressions on
> > >>>> Windows/Linux. These test passed on M2:
> > >>>>  org.apache.geronimo.tomcat.ContainerTest
> > >>>>  org.apache.geronimo.tomcat.JAASSecurityTest
> > >>>>
> > >>>> - JDKTools Tests
> > >>>>  1 failure on Windows. The test stably fails when it runs in a
bunch
> > >>>> with others and may fail if run standalone :
> > >>>>
> > >>>> - Reliability Test Suite
> > >>>>  The status is a bit unclear to me. There is 1 test is incorrect
> > >>>> test. And some tests failed because of timeout but its increase
didn't
> > >>>> help.
> > >>>>  Also some tests were removed from exclude list for r579330 but
some
> > >>>> of them failed.
> > >>>>
> > >>>> - Stress Test Suite
> > >>>>  3 failures on Windows and 11 on Linux - I haven't seen analysis
report
> > >>>>
> > >>>> - VTS VM Test Suite
> > >>>>  1 failure on Linux - the test passed on previous snapshot should
be analysed.
> > >>>>
> > >>>> Any comments are welcome.
> > >>>>
> > >>>> [1] http://people.apache.org/~mloenko/snapshot_testing/script/r579330/index.html
> > >>>>
> > >>>> Thanks,
> > >>>> Stepan Mishura
> > >>>> Intel Enterprise Solutions Software Division
>

Mime
View raw message