harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pavel Ozhdikhin" <pavel.ozhdik...@gmail.com>
Subject Re: [drlvm][unit] 100% of class library tests pass
Date Thu, 16 Nov 2006 10:40:06 GMT
Sorry to say but it actually does not work until there is no notifications
to the mailing list and no immediate reaction to the regressions.

thanks,
Pavel


On 11/16/06, Fedotov, Alexei A <alexei.a.fedotov@intel.com> wrote:
>
> Pavel, All,
>
> Let me add one "pro" for the second approach: it works already.
> Vladimir's scripts daily upload test results to http://harmonytest.org.
>
> With best regards,
> Alexei Fedotov,
> Intel Java & XML Engineering
>
> >-----Original Message-----
> >From: Tim Ellison [mailto:t.p.ellison@gmail.com]
> >Sent: Thursday, November 16, 2006 12:37 PM
> >To: harmony-dev@incubator.apache.org
> >Subject: Re: [drlvm][unit] 100% of class library tests pass
> >
> >Pavel Ozhdikhin wrote:
> >> We have to evolving systems - classlib and DRLVM. To check commits to
> >> classlib we need a stable DRLVM which can pass 100% of HUT. Otherwise
> >it's
> >> impossible to use DRLVM for pre-commit testing - you never know
> whether
> >> your
> >> test fail because of your patch or due to latest changes in DRLVM.
> >>
> >> I remember the time when DRLVM and Jitrino actively evolved - for
> some
> >time
> >> JIT had to use an older version of DRLVM which could pass all commit
> >> criteria because newer versions suffered from regressions. And
> finally we
> >> came to comon strict commit criteria which prevented regressions in
> both
> >VM
> >> and JIT.
> >>
> >> To avoid regressions using DRLVM in classlib testing I see 3 possible
> >> solutions:
> >>
> >> 1. Use one fixed DRLVM version which can pass 100% HUT test. Update
> this
> >> version from time to time.
> >>    Pros: + Less time to run DRLVM pre-commit tests
> >>              + Classlib does not suffer from regressions in DRLVM
> >>    Cons: - DRLVM will suffer from regressions
> >>               - Classlib can not use the latest DRLVM
> >>               - Need additional efforts to regain stability on DRLVM
> >>                 when we want to update the version for classlib
> testing
> >>
> >> 2. Add HUT to CruiseControl testing on DRLVM and rollback commits
> causing
> >> regressions
> >>    Pros: + Less time to run DRLVM pre-commit tests
> >>              + Classlib can use the latest DRLVM
> >>    Cons: - Classlib can suffer from DRLVM regressions (time lag
> before
> >> rollback)
> >>               - It is not always clear which commit caused a
> regression
> >>               - Rollbacks are costly
> >>
> >> 3. Add HUT to the commit criteria for DRLVM
> >>     Pros: + Classlib always can use the latest DRLVM
> >>               + DRLVM has no regressions regarding to HUT
> >>     Cons: - More time to run DRLVM pre-commit tests (I was told that
> HUT
> >>                   take 25 minutes running in single JVM mode)
> >>
> >> I think that preventing a problem is better than solving it
> afterwards.
> >So,
> >> I personally would choose the 3rd approach, don't mind against the
> second
> >> and dislike the first one. Probably some combination of these is
> >possible.
> >
> >While I appreciate the desire to keep things stable, I think it is
> >unreasonable to ask developers to run the entire test suite each time.
> >As we have seen in the classlib code, running targeted tests before
> >commit and leaving the build machines to run continuous tests ensures
> >that we are productive and are notified of breakages in time to easily
> >back out a patch and re-evaluate.
> >
> >With the amount of machine time we have running harmony tests on
> >different cpu's/os's/compilers/etc we are getting better coverage than
> >any individual could be expected to provide.
> >
> >Which is a long way of saying I think option (2) above is best -- and
> >relies on the bid machines letting us know if things break, and the
> >commitment from all of us to go straight in and fix it.
> >
> >Regards,
> >Tim
> >
> >--
> >
> >Tim Ellison (t.p.ellison@gmail.com)
> >IBM Java technology centre, UK.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message