harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Beliaev" <vladimir.k.beli...@gmail.com>
Subject Re: [general][M3] Eclipse unit tests 3.2 result on Windows_x86 for r579330 (was: Re: [general] M3 - code frozen)
Date Fri, 28 Sep 2007 18:19:00 GMT
I've attached patches for:

HARMONY-4870 <https://issues.apache.org/jira/browse/HARMONY-4870> [fixed EFL
for EUT3.2]
HARMONY-4871 <https://issues.apache.org/jira/browse/HARMONY-4871> [fixed EFL
for EUT3.3]
HARMONY-4878 <https://issues.apache.org/jira/browse/HARMONY-4878>
[buildtest][eut]
copy EFL to report on people.apache.org]

*Stepan*, could you get them committed & integrated to BTI to see the
recaculated EUT status on M3 snapshot (r579330), *please*?

Thanks
Vladimir

2007/9/28, Ilya Berezhniuk <ilya.berezhniuk@gmail.com>:
>
> Vladimir, Stepan,
>
> Together with adding HARMONY-4822 to EFL, I suggest excluding
> HARMONY-3361 and HARMONY-4191 from both EFLs (the first bug is not
> reproducible anymore and the second one is fixed already).
> Also HARMONY-3361 and HARMONY-3382 can now be closed as not reproducible.
>
> --
>
> Ilya
>
>
> 2007/9/27, Vladimir Beliaev <vladimir.k.beliaev@gmail.com>:
> > Hello, Ilya, Stepan,
> >
> > yes, I confirm the Expected Failure List was not updated yes, so the *
> > jdtcoremodel* failure is actually expected.
> >
> > I also confirm I did not reproduce 3 new issues in *coreresources*
> suite.
> > They also not reported in previous CC-results. So these may be
> intermittent
> > failures.
> >
> > And *coreresources* does have dependency from D:\temp directory
> existance.
> > I've got 3 issues on machine w/o drive D:. And I double checked then
> that
> > these 3 issues are not reproducible on machine with D: drive (seems to
> be
> > EUT issues).
> >
> > Thanks
> > Vladimir
> >
> > P.S. I'll send the patch for EFL update soon.
> >
> > 2007/9/27, Ilya Berezhniuk <ilya.berezhniuk@gmail.com>:
> > >
> > > I missed that testDeleteMultipleMembersFromVariousCUs test was not
> > > added to EFL, and looked at wrong test in jdtcoremodel...
> > > So there are only 3 actually unexpected failures which I still can't
> > > reproduce...
> > >
> > > --
> > >
> > > Ilya
> > >
> > > 2007/9/27, Ilya Berezhniuk <ilya.berezhniuk@gmail.com>:
> > > > Hello Stepan,
> > > >
> > > > I tried to reproduce latest EUT 3.2/Windows results for r579330.
> > > > The results are the following:
> > > > - single unexpected failure in jdtcoremodel is stably reproducible
> on
> > > > separate suite; I'm now investigating this.
> > > > - 3 failures in coreresources suite did not appear on both separate
> > > > suite run and whole suite run.
> > > >
> > > > AFAIK, Vladimir also reproduced jdtcoremodel failure and cannot
> > > > reproduce those 3 failures. He got another failures because EUT
> tried
> > > > to create temporary files on CDROM :)
> > > >
> > > >
> > > > 2007/9/26, Vladimir Beliaev <vladimir.k.beliaev@gmail.com>:
> > > > > > Is it possible to resolve it by simply increasing timeout?
> > > > >
> > > > > no, timeout does not help here:
> > > > >
> > > > > 1. this would increase the EUT running time from 8h to ...
> infinity
> > > (which
> > > > > is not helpful at all for daily testing of EUT)
> > > > >
> > > > > 2. I have an expirience of such slow jdtcorecompiler running - as
> a
> > > result
> > > > > the suite did not crash still the run took 4-6h (instead of 30-50
> min)
> > > and
> > > > > half of 6'000 tests ended with error/failure - this is not
> acceptable
> > > > > because these failures / error are still not reproducible on other
> > > hosts, so
> > > > > nothing in Harmony code to be fixed (still host configuration is
> to be
> > > > > fixed).
> > > > >
> > > > > I can send an instruction of running
> > > > > org.eclipse.jdt.core.tests.compiler.regression.TestAll separately
> (w/o
> > > other
> > > > > jdtcorecompile JUnit suites). Then you may try to run this EUT
> Suite
> > > on
> > > > > other machine to track the running speed.
> > > > >
> > > > > I have no idea of what may cause this suite running slow on
> particular
> > > host.
> > > > > May be the disk is close to be dead (still this would affect other
> > > suites as
> > > > > weel not only jdtcorecompiler one).
> > > > >
> > > > > Thanks
> > > > > Vladimir Beliaev
> > > > >
> > > > > 2007/9/26, Stepan Mishura <stepan.mishura@gmail.com>:
> > > > > >
> > > > > > On 9/26/07, Vladimir Beliaev <vladimir.k.beliaev@gmail.com>
> wrote:
> > > > > > > Hello, Stepan,
> > > > > > >
> > > > > > > 1. org.eclipse.jdt.core.tests.compiler.regression.TestAll
-
> the
> > > issues
> > > > > > is
> > > > > > > not reproduced locally. I'm looking to output.txt
> > > > > > > <
> > > http://people.apache.org/~smishura/r579330/Linux_x86/eut/output.txt
> > > > > > >from
> > > > > > > EUT3.2/Linux run - it says this suite was killed by timeout:
> > > > > > >
> > > > > >
> > > > > > Is it possible to resolve it by simply increasing timeout?
> > > > > >
> > > > > > -Stepan.
> > > > > >
> > > > > > >    eclipse-test:
> > > > > > >         [echo] *Running
> > > > > > > org.eclipse.jdt.core.tests.compiler.regression.TestAll*
> > > > > > >         [java] Apache Harmony Launcher : (c) Copyright
1991,
> 2006
> > > The
> > > > > > > Apache Software Foundation or its licensors, as applicable.
> > > > > > >         [java] java version "1.5.0"
> > > > > > >         [java] pre-alpha : not complete or compatible
> > > > > > >         [java] svn = r579330, (Sep 26 2007), Linux/ia32/gcc
> 3.3.3,
> > > > > > release
> > > > > > > build
> > > > > > >         [java] http://harmony.apache.org
> > > > > > >         [java] *Timeout: killed the sub-process*
> > > > > > >
> > > > > > > So looks like a host configyuration issues again...
> > > > > > >
> > > > > > > > Am I missing something?
> > > > > > >
> > > > > > > Here is a list of "configuration tricks" I know for EUT
on
> > > Linux/x86
> > > > > > (some
> > > > > > > of them were noted by you to me):
> > > > > > >
> > > > > > > 1. the test host *must not be load *(CPU/disk must be used
by
> EUT
> > > only)
> > > > > > -
> > > > > > > EUT is sensetive to timeout.
> > > > > > > 2. DISPLAY is set to other Linux host with pure X-server
(not
> VNC,
> > > > > > cygwin
> > > > > > > X-server, Xvfb, etc.)
> > > > > > > 3. -Duser.home=<my tmp-home with .subversion directory
copied
> from
> > > ~/.>
> > > > > > (no
> > > > > > > $HOME file collisions should happen)
> > > > > > > 4. -Djava.io.tmpdir=<my tmp dir> (no $TMP file collisions
> should
> > > happen)
> > > > > > > 5. ulimit –n 65'536 (max number of open file handlers)
> > > > > > > 6. mozilla is installed
> > > > > > > 6.1 export MOZILLA_FIVE_HOME=/opt/mozilla/lib
> > > > > > > 6.2 export LD_LIBRARY_PATH=/opt/mozilla/lib
> > > > > > >
> > > > > > > I'm still not sure this would help with slow running of
> > > > > > > *jdtcorecompiler*suite. Let's try to get it resolved together.
> > > > > > >
> > > > > > > Thanks
> > > > > > > Vladimir Beliaev
> > > > > > >
> > > > > > > 2007/9/26, Stepan Mishura <stepan.mishura@gmail.com>:
> > > > > > > >
> > > > > > > > On 9/25/07, Stepan Mishura wrote:
> > > > > > > > <SNIP>
> > > > > > > > > I've compared testing status for the last snapshot
r578410
> [1]
> > > with
> > > > > > M2
> > > > > > > > [2][3].
> > > > > > > > >
> > > > > > > > > We have the following short status for failed
suites:
> > > > > > > > >
> > > > > > > > > Windows_x86:
> > > > > > > > >  * classlib: 2 tests from pack200 module fail
on snapshot
> (but
> > > pass
> > > > > > > > > on debug build)
> > > > > > > > >  * Eclipse unit tests 3.2: there is no tests
report like
> for
> > > M3. The
> > > > > > > > > pass rate was improved from 99.32%[3] to 99.7%[1]
> > > > > > > > >  * Eclipse unit tests 3.3 are new and the pass
rate is
> 99.77%.
> > > I
> > > > > > > > > think is acceptable
> > > > > > > > >  * EGA x48 hours scenario fails. According to
[4] it
> passed on
> > > M2.
> > > > > > > > >  * Functional: need more analysis, currently
I see that 2
> > > tests were
> > > > > > > > > enabled and new 15 regressions.
> > > > > > > > >  * Geronimo: 2 regressions
> > > > > > > > >  * JDK tools: 1 test failed. It might be intermediate
> failure
> > > - the
> > > > > > > > > test failed due to timeout
> > > > > > > > >  * Reliability: 65 tests passed for M2 and 64
for M3.
> > > Investigation
> > > > > > > > > is required.
> > > > > > > > >  * Stress: 190 tests passed for M2 and 189 for
M3.
> > > Investigation is
> > > > > > > > required.
> > > > > > > > >
> > > > > > > > > Linux_x86:
> > > > > > > > >  * classlib: 2 tests from pack200 (as for Windows),
 1
> luni
> > > tests
> > > > > > > > > failed and 1 crash of security test
> > > > > > > > >  * Eclipse unit tests 3.2: 2 suites crashed so
pass rate
> is
> > > 69.60%.
> > > > > > I
> > > > > > > > > assume this may be CC host configuration issue.
Going to
> > > > > > investigate.
> > > > > > > >
> > > > > > > > I'm looking into EUT 3.2 results on Linux_x86 for
r579330.
> > > > > > > > The good news that crashes for the next suites disappeared:
> > > > > > > > - org.eclipse.jdt.core.tests.dom.RunAllTests
> > > > > > > > - org.eclipse.jdt.core.tests.model.AllJavaModelTests
> > > > > > > >
> > > > > > > > And the bad news there is one new suite crash:
> > > > > > > > - org.eclipse.jdt.core.tests.compiler.regression.TestAll
> > > > > > > >
> > > > > > > > This results are pretty confusing for me. I believe
that
> last
> > > updates
> > > > > > > > to classlib/drlvm/jdktool shouldn't influence on results
in
> such
> > > > > > > > dramatic way.
> > > > > > > >
> > > > > > > > I have to admit that CC host configuration was changed
a
> bit.
> > > There is
> > > > > > > > an assumption that the tests are sensitive to X-server.
> > > (currently VNC
> > > > > > > > is used). So X-server (SLES9) was configured on CC
host and
> > > runlevel
> > > > > > > > was changed from 3 to 5. But IMHO it shouldn't affect
> current CC
> > > > > > > > configuration (I didn't change it. i.e. VNC was used
as
> usual)
> > > and
> > > > > > > > testing result as well.
> > > > > > > >
> > > > > > > > Am I missing something?
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Stepan.
> > > > > > > >
> > > > > > > > >  * Eclipse unit tests 3.3 are new and the pass
rate is
> 96.47%.
> > > I
> > > > > > > > > think is acceptable
> > > > > > > > >  * EGA x48 hours scenario: the same as for Windows
> (scenario
> > > fails
> > > > > > on
> > > > > > > > M3)
> > > > > > > > >  * Functional: need more analysis, similar to
Windows -
> some
> > > test
> > > > > > are
> > > > > > > > > passing now but there are new failures.
> > > > > > > > >  * Geronimo: 2 regressions (the same as for Windows)
> > > > > > > > >  * Reliability: need more analysis
> > > > > > > > >  * Stress: need more analysis
> > > > > > > > >
> > > > > > > > > As I remember Sean took pack200 tests. And Alexei
Zakharov
> > > took
> > > > > > > > > security test crash.
> > > > > > > > > I'm going to sort things out with Eclipse unit
tests 3.2crash
> > > on
> > > > > > > > > Linux. And to look info failed jdktools test.
> > > > > > > > >
> > > > > > > > > So volunteers are required for: EGAx48, Geronimo,
> functional,
> > > > > > > > > reliability and stress suites.
> > > > > > > > >
> > > > > > > > > Also we have 2 JIRAs to be resolved for M3 under
milstone
> > > > > > unmblella[5]:
> > > > > > > > > https://issues.apache.org/jira/browse/HARMONY-4844
> > > > > > > > > https://issues.apache.org/jira/browse/HARMONY-4810
> > > > > > > > >
> > > > > > > > > [1]
> > > > > > > >
> > > > > >
> > >
> http://people.apache.org/~mloenko/snapshot_testing/script/r578410/index.html
> > > > > > > > > [2]
> > > > > > > >
> > > > > >
> > >
> http://people.apache.org/~mloenko/snapshot_testing/script/r551077/index.html
> > > > > > > > > [3] http://wiki.apache.org/harmony/milestones/M2
> > > > > > > > > [4]
> > > > > > > >
> > > > > >
> > >
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200706.mbox/%3c678b3f320706290508l554079dau6af1a82b17d62b54@mail.gmail.com%3e
> > > > > > > > > [5] https://issues.apache.org/jira/browse/HARMONY-4843
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Stepan.
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Ilya
> > > >
> > >
> >
> >
> >
> > --
> > Vladimir Beliaev
> > Intel Middleware Products Division
> >
>



-- 
Vladimir Beliaev
Intel Middleware Products Division

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message