harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Ivanov" <ivavladi...@gmail.com>
Subject Re: [drlvm][unit] 100% of class library tests pass
Date Thu, 16 Nov 2006 10:59:42 GMT
On 11/16/06, Pavel Ozhdikhin <pavel.ozhdikhin@gmail.com> wrote:
>
> Sorry to say but it actually does not work until there is no notifications
> to the mailing list and no immediate reaction to the regressions.


Actually, no regression was detected today so no notifications were sent.

 thanks, Vladimir

thanks,
> Pavel
>
>
> On 11/16/06, Fedotov, Alexei A <alexei.a.fedotov@intel.com> wrote:
> >
> > Pavel, All,
> >
> > Let me add one "pro" for the second approach: it works already.
> > Vladimir's scripts daily upload test results to http://harmonytest.org.
> >
> > With best regards,
> > Alexei Fedotov,
> > Intel Java & XML Engineering
> >
> > >-----Original Message-----
> > >From: Tim Ellison [mailto:t.p.ellison@gmail.com]
> > >Sent: Thursday, November 16, 2006 12:37 PM
> > >To: harmony-dev@incubator.apache.org
> > >Subject: Re: [drlvm][unit] 100% of class library tests pass
> > >
> > >Pavel Ozhdikhin wrote:
> > >> We have to evolving systems - classlib and DRLVM. To check commits to
> > >> classlib we need a stable DRLVM which can pass 100% of HUT. Otherwise
> > >it's
> > >> impossible to use DRLVM for pre-commit testing - you never know
> > whether
> > >> your
> > >> test fail because of your patch or due to latest changes in DRLVM.
> > >>
> > >> I remember the time when DRLVM and Jitrino actively evolved - for
> > some
> > >time
> > >> JIT had to use an older version of DRLVM which could pass all commit
> > >> criteria because newer versions suffered from regressions. And
> > finally we
> > >> came to comon strict commit criteria which prevented regressions in
> > both
> > >VM
> > >> and JIT.
> > >>
> > >> To avoid regressions using DRLVM in classlib testing I see 3 possible
> > >> solutions:
> > >>
> > >> 1. Use one fixed DRLVM version which can pass 100% HUT test. Update
> > this
> > >> version from time to time.
> > >>    Pros: + Less time to run DRLVM pre-commit tests
> > >>              + Classlib does not suffer from regressions in DRLVM
> > >>    Cons: - DRLVM will suffer from regressions
> > >>               - Classlib can not use the latest DRLVM
> > >>               - Need additional efforts to regain stability on DRLVM
> > >>                 when we want to update the version for classlib
> > testing
> > >>
> > >> 2. Add HUT to CruiseControl testing on DRLVM and rollback commits
> > causing
> > >> regressions
> > >>    Pros: + Less time to run DRLVM pre-commit tests
> > >>              + Classlib can use the latest DRLVM
> > >>    Cons: - Classlib can suffer from DRLVM regressions (time lag
> > before
> > >> rollback)
> > >>               - It is not always clear which commit caused a
> > regression
> > >>               - Rollbacks are costly
> > >>
> > >> 3. Add HUT to the commit criteria for DRLVM
> > >>     Pros: + Classlib always can use the latest DRLVM
> > >>               + DRLVM has no regressions regarding to HUT
> > >>     Cons: - More time to run DRLVM pre-commit tests (I was told that
> > HUT
> > >>                   take 25 minutes running in single JVM mode)
> > >>
> > >> I think that preventing a problem is better than solving it
> > afterwards.
> > >So,
> > >> I personally would choose the 3rd approach, don't mind against the
> > second
> > >> and dislike the first one. Probably some combination of these is
> > >possible.
> > >
> > >While I appreciate the desire to keep things stable, I think it is
> > >unreasonable to ask developers to run the entire test suite each time.
> > >As we have seen in the classlib code, running targeted tests before
> > >commit and leaving the build machines to run continuous tests ensures
> > >that we are productive and are notified of breakages in time to easily
> > >back out a patch and re-evaluate.
> > >
> > >With the amount of machine time we have running harmony tests on
> > >different cpu's/os's/compilers/etc we are getting better coverage than
> > >any individual could be expected to provide.
> > >
> > >Which is a long way of saying I think option (2) above is best -- and
> > >relies on the bid machines letting us know if things break, and the
> > >commitment from all of us to go straight in and fix it.
> > >
> > >Regards,
> > >Tim
> > >
> > >--
> > >
> > >Tim Ellison (t.p.ellison@gmail.com)
> > >IBM Java technology centre, UK.
> >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message