harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrey Yakushev" <andrey.yakus...@gmail.com>
Subject Re: [general] M3 - code frozen
Date Tue, 25 Sep 2007 15:54:34 GMT
I also made some analysis for reliability test suite.
I've found that several tests didn't fail but rather were killed by
guarding timeout.

They are:
api.nio.channels.filechannel.FileChannelMapTest
api.net.DatagramTest
vm.finalization.FinalizeThrowRestoreTest
vm.gc.WeakRefsTest
api.serialization.SerializableClassesTest

I think it is due to slowing down of the test host after repeated
tests iteration. I suggest increasing the killing timeout. I've opened
HARMONY-4851 for this.

I've run excluded reliability tests several times on Windows and Linux
and I don't see the failures in the following excluded tests now:

api.kernel.thread.Calculation.CalcTest
api.kernel.thread.StackTraceTest.StackTraceTest
api.nio.charset.ContainsTest
api.text.DecimalFormat_Locales
api.zip.ZlibTest

I suggest removing them from the exclude list. I've opened
HARMONY-4850 for this.

api.net.HttpConnectionTest fails due to unavailability of
harmony.apache.org site from the test host network. It works fine if
site is accessible.

Other failures seem the real reliability issues. These tests fail only
after several successful iterations.

Thanks,
Andrey




On 9/25/07, Tim Ellison <t.p.ellison@gmail.com> wrote:
> I'll be looking at some of the functional, reliability, and stress tests
> failures and seeing if there is anything I can help with.  I'll post to
> the list for any areas that I pick up.
>
> Regards,
> Tim
>
> Stepan Mishura wrote:
> > On 9/24/07, Stepan Mishura <stepan.mishura@gmail.com> wrote:
> >> On 9/22/07, Stepan Mishura wrote:
> >>> Hi,
> >>>
> >>> According to the plan - Sept. 22 is code freeze date for M3 so let's
> >>> follow policy:
> >>> "no more commits please without agreement from two committers on the dev
list."
> >>>
> >>> Let's do more testing and analyze if there any critical/blocker issues
> >>> for consideration.
> >>> Please raise here issues that you think MUST be fixed for M3.
> >> Here is [1] the status of the last snapshot (r578410) that includes
> >> last classlib updates. Unfortunately, not everything is green there.
> >> I'm going to inspect all results to make sure that there are no
> >> failures caused by CC configuration issues.
> >>
> >
> > I've compared testing status for the last snapshot r578410 [1] with M2 [2][3].
> >
> > We have the following short status for failed suites:
> >
> > Windows_x86:
> >   * classlib: 2 tests from pack200 module fail on snapshot (but pass
> > on debug build)
> >   * Eclipse unit tests 3.2: there is no tests report like for M3. The
> > pass rate was improved from 99.32%[3] to 99.7%[1]
> >   * Eclipse unit tests 3.3 are new and the pass rate is 99.77%. I
> > think is acceptable
> >   * EGA x48 hours scenario fails. According to [4] it passed on M2.
> >   * Functional: need more analysis, currently I see that 2 tests were
> > enabled and new 15 regressions.
> >   * Geronimo: 2 regressions
> >   * JDK tools: 1 test failed. It might be intermediate failure - the
> > test failed due to timeout
> >   * Reliability: 65 tests passed for M2 and 64 for M3. Investigation
> > is required.
> >   * Stress: 190 tests passed for M2 and 189 for M3. Investigation is required.
> >
> > Linux_x86:
> >   * classlib: 2 tests from pack200 (as for Windows),  1 luni tests
> > failed and 1 crash of security test
> >   * Eclipse unit tests 3.2: 2 suites crashed so pass rate is 69.60%. I
> > assume this may be CC host configuration issue. Going to investigate.
> >   * Eclipse unit tests 3.3 are new and the pass rate is 96.47%. I
> > think is acceptable
> >   * EGA x48 hours scenario: the same as for Windows (scenario fails on M3)
> >   * Functional: need more analysis, similar to Windows - some test are
> > passing now but there are new failures.
> >   * Geronimo: 2 regressions (the same as for Windows)
> >   * Reliability: need more analysis
> >   * Stress: need more analysis
> >
> > As I remember Sean took pack200 tests. And Alexei Zakharov took
> > security test crash.
> > I'm going to sort things out with Eclipse unit tests 3.2 crash on
> > Linux. And to look info failed jdktools test.
> >
> > So volunteers are required for: EGAx48, Geronimo, functional,
> > reliability and stress suites.
> >
> > Also we have 2 JIRAs to be resolved for M3 under milstone unmblella[5]:
> > https://issues.apache.org/jira/browse/HARMONY-4844
> > https://issues.apache.org/jira/browse/HARMONY-4810
> >
> > [1] http://people.apache.org/~mloenko/snapshot_testing/script/r578410/index.html
> > [2] http://people.apache.org/~mloenko/snapshot_testing/script/r551077/index.html
> > [3] http://wiki.apache.org/harmony/milestones/M2
> > [4] http://mail-archives.apache.org/mod_mbox/harmony-dev/200706.mbox/%3c678b3f320706290508l554079dau6af1a82b17d62b54@mail.gmail.com%3e
> > [5] https://issues.apache.org/jira/browse/HARMONY-4843
> >
> > Thanks,
> > Stepan.
> >
> >
> >> Please feel free to pick up any issue for investigation. For example,
> >> 2 pack200 classlib test failed. Also there is a crash of
> >> org.apache.harmony.security.tests.java.security.cert.serialization.CertificateTest
> >> on Linux x86
> >>
> >> BTW, should we consider BTI's workspace (that we use for M3 testing)
> >> frozen too?
> >>
> >> [1] http://people.apache.org/~mloenko/snapshot_testing/script/r578410/index.html
> >>
> >> Thanks,
> >> Stepan.
> >
>


-- 
Thanks,
Andrey

Mime
View raw message