Mailing-List: contact dev-help@harmony.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@harmony.apache.org
Received-SPF: pass (athena.apache.org: domain of vladimir.k.beliaev@gmail.com
 designates 64.233.166.183 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=beta;
        h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references;
        b=FwPj/1x65lg0YZw2J8oT323CevsPs5dmVyyGj/dlbHrhKViJBzlNUveOTm6zb1yrG/E7wPibonJS9xNXd0ZWfUE27UJ5HWFBL2CbZwTq3bGHuTcjGnuEwnRIGXKtXLTq07mcENF9I7Axtx0mqp2k/n3NQlmMunqoVUR5fjmQROc=
Message-ID: <587698b80709260306i35967395w41e0d431f1ed0d3c@mail.gmail.com>
Date: Wed, 26 Sep 2007 14:06:48 +0400
From: "Vladimir Beliaev" <vladimir.k.beliaev@gmail.com>
To: dev@harmony.apache.org
Subject: Re: [general][M3] Update for Eclipse unit tests 3.2 testing result on
 Linux_x86 (was: Re: [general] M3 - code frozen)
In-Reply-To: <6e47b64f0709260255s721dd891u7870d6996f931e07@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_6287_6775011.1190801208350"
References: <6e47b64f0709260141q123d569dn9d659a2569540e1b@mail.gmail.com>
	 <587698b80709260202i106638f7h9d09d4d2412c5da5@mail.gmail.com>
	 <6e47b64f0709260255s721dd891u7870d6996f931e07@mail.gmail.com>

------=_Part_6287_6775011.1190801208350
Content-Type: text/plain; charset=WINDOWS-1252
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

> Is it possible to resolve it by simply increasing timeout?

no, timeout does not help here:

1. this would increase the EUT running time from 8h to ... infinity (which
is not helpful at all for daily testing of EUT)

2. I have an expirience of such slow jdtcorecompiler running - as a result
the suite did not crash still the run took 4-6h (instead of 30-50 min) and
half of 6'000 tests ended with error/failure - this is not acceptable
because these failures / error are still not reproducible on other hosts, s=
o
nothing in Harmony code to be fixed (still host configuration is to be
fixed).

I can send an instruction of running
org.eclipse.jdt.core.tests.compiler.regression.TestAll separately (w/o othe=
r
jdtcorecompile JUnit suites). Then you may try to run this EUT Suite on
other machine to track the running speed.

I have no idea of what may cause this suite running slow on particular host=
.
May be the disk is close to be dead (still this would affect other suites a=
s
weel not only jdtcorecompiler one).

Thanks
Vladimir Beliaev

2007/9/26, Stepan Mishura <stepan.mishura@gmail.com>:
>
> On 9/26/07, Vladimir Beliaev <vladimir.k.beliaev@gmail.com> wrote:
> > Hello, Stepan,
> >
> > 1. org.eclipse.jdt.core.tests.compiler.regression.TestAll - the issues
> is
> > not reproduced locally. I'm looking to output.txt
> > <http://people.apache.org/~smishura/r579330/Linux_x86/eut/output.txt
> >from
> > EUT3.2/Linux run - it says this suite was killed by timeout:
> >
>
> Is it possible to resolve it by simply increasing timeout?
>
> -Stepan.
>
> >    eclipse-test:
> >         [echo] *Running
> > org.eclipse.jdt.core.tests.compiler.regression.TestAll*
> >         [java] Apache Harmony Launcher : (c) Copyright 1991, 2006 The
> > Apache Software Foundation or its licensors, as applicable.
> >         [java] java version "1.5.0"
> >         [java] pre-alpha : not complete or compatible
> >         [java] svn =3D r579330, (Sep 26 2007), Linux/ia32/gcc 3.3.3,
> release
> > build
> >         [java] http://harmony.apache.org
> >         [java] *Timeout: killed the sub-process*
> >
> > So looks like a host configyuration issues again...
> >
> > > Am I missing something?
> >
> > Here is a list of "configuration tricks" I know for EUT on Linux/x86
> (some
> > of them were noted by you to me):
> >
> > 1. the test host *must not be load *(CPU/disk must be used by EUT only)
> -
> > EUT is sensetive to timeout.
> > 2. DISPLAY is set to other Linux host with pure X-server (not VNC,
> cygwin
> > X-server, Xvfb, etc.)
> > 3. -Duser.home=3D<my tmp-home with .subversion directory copied from ~/=
.>
> (no
> > $HOME file collisions should happen)
> > 4. -Djava.io.tmpdir=3D<my tmp dir> (no $TMP file collisions should happ=
en)
> > 5. ulimit =96n 65'536 (max number of open file handlers)
> > 6. mozilla is installed
> > 6.1 export MOZILLA_FIVE_HOME=3D/opt/mozilla/lib
> > 6.2 export LD_LIBRARY_PATH=3D/opt/mozilla/lib
> >
> > I'm still not sure this would help with slow running of
> > *jdtcorecompiler*suite. Let's try to get it resolved together.
> >
> > Thanks
> > Vladimir Beliaev
> >
> > 2007/9/26, Stepan Mishura <stepan.mishura@gmail.com>:
> > >
> > > On 9/25/07, Stepan Mishura wrote:
> > > <SNIP>
> > > > I've compared testing status for the last snapshot r578410 [1] with
> M2
> > > [2][3].
> > > >
> > > > We have the following short status for failed suites:
> > > >
> > > > Windows_x86:
> > > >  * classlib: 2 tests from pack200 module fail on snapshot (but pass
> > > > on debug build)
> > > >  * Eclipse unit tests 3.2: there is no tests report like for M3. Th=
e
> > > > pass rate was improved from 99.32%[3] to 99.7%[1]
> > > >  * Eclipse unit tests 3.3 are new and the pass rate is 99.77%. I
> > > > think is acceptable
> > > >  * EGA x48 hours scenario fails. According to [4] it passed on M2.
> > > >  * Functional: need more analysis, currently I see that 2 tests wer=
e
> > > > enabled and new 15 regressions.
> > > >  * Geronimo: 2 regressions
> > > >  * JDK tools: 1 test failed. It might be intermediate failure - the
> > > > test failed due to timeout
> > > >  * Reliability: 65 tests passed for M2 and 64 for M3. Investigation
> > > > is required.
> > > >  * Stress: 190 tests passed for M2 and 189 for M3. Investigation is
> > > required.
> > > >
> > > > Linux_x86:
> > > >  * classlib: 2 tests from pack200 (as for Windows),  1 luni tests
> > > > failed and 1 crash of security test
> > > >  * Eclipse unit tests 3.2: 2 suites crashed so pass rate is 69.60%.
> I
> > > > assume this may be CC host configuration issue. Going to
> investigate.
> > >
> > > I'm looking into EUT 3.2 results on Linux_x86 for r579330.
> > > The good news that crashes for the next suites disappeared:
> > > - org.eclipse.jdt.core.tests.dom.RunAllTests
> > > - org.eclipse.jdt.core.tests.model.AllJavaModelTests
> > >
> > > And the bad news there is one new suite crash:
> > > - org.eclipse.jdt.core.tests.compiler.regression.TestAll
> > >
> > > This results are pretty confusing for me. I believe that last updates
> > > to classlib/drlvm/jdktool shouldn't influence on results in such
> > > dramatic way.
> > >
> > > I have to admit that CC host configuration was changed a bit. There i=
s
> > > an assumption that the tests are sensitive to X-server. (currently VN=
C
> > > is used). So X-server (SLES9) was configured on CC host and runlevel
> > > was changed from 3 to 5. But IMHO it shouldn't affect current CC
> > > configuration (I didn't change it. i.e. VNC was used as usual) and
> > > testing result as well.
> > >
> > > Am I missing something?
> > >
> > > Thanks,
> > > Stepan.
> > >
> > > >  * Eclipse unit tests 3.3 are new and the pass rate is 96.47%. I
> > > > think is acceptable
> > > >  * EGA x48 hours scenario: the same as for Windows (scenario fails
> on
> > > M3)
> > > >  * Functional: need more analysis, similar to Windows - some test
> are
> > > > passing now but there are new failures.
> > > >  * Geronimo: 2 regressions (the same as for Windows)
> > > >  * Reliability: need more analysis
> > > >  * Stress: need more analysis
> > > >
> > > > As I remember Sean took pack200 tests. And Alexei Zakharov took
> > > > security test crash.
> > > > I'm going to sort things out with Eclipse unit tests 3.2 crash on
> > > > Linux. And to look info failed jdktools test.
> > > >
> > > > So volunteers are required for: EGAx48, Geronimo, functional,
> > > > reliability and stress suites.
> > > >
> > > > Also we have 2 JIRAs to be resolved for M3 under milstone
> unmblella[5]:
> > > > https://issues.apache.org/jira/browse/HARMONY-4844
> > > > https://issues.apache.org/jira/browse/HARMONY-4810
> > > >
> > > > [1]
> > >
> http://people.apache.org/~mloenko/snapshot_testing/script/r578410/index.h=
tml
> > > > [2]
> > >
> http://people.apache.org/~mloenko/snapshot_testing/script/r551077/index.h=
tml
> > > > [3] http://wiki.apache.org/harmony/milestones/M2
> > > > [4]
> > >
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200706.mbox/%3c678b3=
f320706290508l554079dau6af1a82b17d62b54@mail.gmail.com%3e
> > > > [5] https://issues.apache.org/jira/browse/HARMONY-4843
> > > >
> > > > Thanks,
> > > > Stepan.
>

------=_Part_6287_6775011.1190801208350--