harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Beliaev" <vladimir.k.beli...@gmail.com>
Subject Re: [general][M3] Update for Eclipse unit tests 3.2 testing result on Linux_x86 (was: Re: [general] M3 - code frozen)
Date Wed, 26 Sep 2007 10:06:48 GMT
> Is it possible to resolve it by simply increasing timeout?

no, timeout does not help here:

1. this would increase the EUT running time from 8h to ... infinity (which
is not helpful at all for daily testing of EUT)

2. I have an expirience of such slow jdtcorecompiler running - as a result
the suite did not crash still the run took 4-6h (instead of 30-50 min) and
half of 6'000 tests ended with error/failure - this is not acceptable
because these failures / error are still not reproducible on other hosts, so
nothing in Harmony code to be fixed (still host configuration is to be
fixed).

I can send an instruction of running
org.eclipse.jdt.core.tests.compiler.regression.TestAll separately (w/o other
jdtcorecompile JUnit suites). Then you may try to run this EUT Suite on
other machine to track the running speed.

I have no idea of what may cause this suite running slow on particular host.
May be the disk is close to be dead (still this would affect other suites as
weel not only jdtcorecompiler one).

Thanks
Vladimir Beliaev

2007/9/26, Stepan Mishura <stepan.mishura@gmail.com>:
>
> On 9/26/07, Vladimir Beliaev <vladimir.k.beliaev@gmail.com> wrote:
> > Hello, Stepan,
> >
> > 1. org.eclipse.jdt.core.tests.compiler.regression.TestAll - the issues
> is
> > not reproduced locally. I'm looking to output.txt
> > <http://people.apache.org/~smishura/r579330/Linux_x86/eut/output.txt
> >from
> > EUT3.2/Linux run - it says this suite was killed by timeout:
> >
>
> Is it possible to resolve it by simply increasing timeout?
>
> -Stepan.
>
> >    eclipse-test:
> >         [echo] *Running
> > org.eclipse.jdt.core.tests.compiler.regression.TestAll*
> >         [java] Apache Harmony Launcher : (c) Copyright 1991, 2006 The
> > Apache Software Foundation or its licensors, as applicable.
> >         [java] java version "1.5.0"
> >         [java] pre-alpha : not complete or compatible
> >         [java] svn = r579330, (Sep 26 2007), Linux/ia32/gcc 3.3.3,
> release
> > build
> >         [java] http://harmony.apache.org
> >         [java] *Timeout: killed the sub-process*
> >
> > So looks like a host configyuration issues again...
> >
> > > Am I missing something?
> >
> > Here is a list of "configuration tricks" I know for EUT on Linux/x86
> (some
> > of them were noted by you to me):
> >
> > 1. the test host *must not be load *(CPU/disk must be used by EUT only)
> -
> > EUT is sensetive to timeout.
> > 2. DISPLAY is set to other Linux host with pure X-server (not VNC,
> cygwin
> > X-server, Xvfb, etc.)
> > 3. -Duser.home=<my tmp-home with .subversion directory copied from ~/.>
> (no
> > $HOME file collisions should happen)
> > 4. -Djava.io.tmpdir=<my tmp dir> (no $TMP file collisions should happen)
> > 5. ulimit –n 65'536 (max number of open file handlers)
> > 6. mozilla is installed
> > 6.1 export MOZILLA_FIVE_HOME=/opt/mozilla/lib
> > 6.2 export LD_LIBRARY_PATH=/opt/mozilla/lib
> >
> > I'm still not sure this would help with slow running of
> > *jdtcorecompiler*suite. Let's try to get it resolved together.
> >
> > Thanks
> > Vladimir Beliaev
> >
> > 2007/9/26, Stepan Mishura <stepan.mishura@gmail.com>:
> > >
> > > On 9/25/07, Stepan Mishura wrote:
> > > <SNIP>
> > > > I've compared testing status for the last snapshot r578410 [1] with
> M2
> > > [2][3].
> > > >
> > > > We have the following short status for failed suites:
> > > >
> > > > Windows_x86:
> > > >  * classlib: 2 tests from pack200 module fail on snapshot (but pass
> > > > on debug build)
> > > >  * Eclipse unit tests 3.2: there is no tests report like for M3. The
> > > > pass rate was improved from 99.32%[3] to 99.7%[1]
> > > >  * Eclipse unit tests 3.3 are new and the pass rate is 99.77%. I
> > > > think is acceptable
> > > >  * EGA x48 hours scenario fails. According to [4] it passed on M2.
> > > >  * Functional: need more analysis, currently I see that 2 tests were
> > > > enabled and new 15 regressions.
> > > >  * Geronimo: 2 regressions
> > > >  * JDK tools: 1 test failed. It might be intermediate failure - the
> > > > test failed due to timeout
> > > >  * Reliability: 65 tests passed for M2 and 64 for M3. Investigation
> > > > is required.
> > > >  * Stress: 190 tests passed for M2 and 189 for M3. Investigation is
> > > required.
> > > >
> > > > Linux_x86:
> > > >  * classlib: 2 tests from pack200 (as for Windows),  1 luni tests
> > > > failed and 1 crash of security test
> > > >  * Eclipse unit tests 3.2: 2 suites crashed so pass rate is 69.60%.
> I
> > > > assume this may be CC host configuration issue. Going to
> investigate.
> > >
> > > I'm looking into EUT 3.2 results on Linux_x86 for r579330.
> > > The good news that crashes for the next suites disappeared:
> > > - org.eclipse.jdt.core.tests.dom.RunAllTests
> > > - org.eclipse.jdt.core.tests.model.AllJavaModelTests
> > >
> > > And the bad news there is one new suite crash:
> > > - org.eclipse.jdt.core.tests.compiler.regression.TestAll
> > >
> > > This results are pretty confusing for me. I believe that last updates
> > > to classlib/drlvm/jdktool shouldn't influence on results in such
> > > dramatic way.
> > >
> > > I have to admit that CC host configuration was changed a bit. There is
> > > an assumption that the tests are sensitive to X-server. (currently VNC
> > > is used). So X-server (SLES9) was configured on CC host and runlevel
> > > was changed from 3 to 5. But IMHO it shouldn't affect current CC
> > > configuration (I didn't change it. i.e. VNC was used as usual) and
> > > testing result as well.
> > >
> > > Am I missing something?
> > >
> > > Thanks,
> > > Stepan.
> > >
> > > >  * Eclipse unit tests 3.3 are new and the pass rate is 96.47%. I
> > > > think is acceptable
> > > >  * EGA x48 hours scenario: the same as for Windows (scenario fails
> on
> > > M3)
> > > >  * Functional: need more analysis, similar to Windows - some test
> are
> > > > passing now but there are new failures.
> > > >  * Geronimo: 2 regressions (the same as for Windows)
> > > >  * Reliability: need more analysis
> > > >  * Stress: need more analysis
> > > >
> > > > As I remember Sean took pack200 tests. And Alexei Zakharov took
> > > > security test crash.
> > > > I'm going to sort things out with Eclipse unit tests 3.2 crash on
> > > > Linux. And to look info failed jdktools test.
> > > >
> > > > So volunteers are required for: EGAx48, Geronimo, functional,
> > > > reliability and stress suites.
> > > >
> > > > Also we have 2 JIRAs to be resolved for M3 under milstone
> unmblella[5]:
> > > > https://issues.apache.org/jira/browse/HARMONY-4844
> > > > https://issues.apache.org/jira/browse/HARMONY-4810
> > > >
> > > > [1]
> > >
> http://people.apache.org/~mloenko/snapshot_testing/script/r578410/index.html
> > > > [2]
> > >
> http://people.apache.org/~mloenko/snapshot_testing/script/r551077/index.html
> > > > [3] http://wiki.apache.org/harmony/milestones/M2
> > > > [4]
> > >
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200706.mbox/%3c678b3f320706290508l554079dau6af1a82b17d62b54@mail.gmail.com%3e
> > > > [5] https://issues.apache.org/jira/browse/HARMONY-4843
> > > >
> > > > Thanks,
> > > > Stepan.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message