harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexey Petrenko" <alexey.a.petre...@gmail.com>
Subject Re: [general] aiming no regression
Date Fri, 01 Dec 2006 10:15:55 GMT
2006/12/1, Rana Dasgupta <rdasgupt@gmail.com>:
> Mikhail,
>   Please see below...
> On 11/30/06, Mikhail Loenko <mloenko@gmail.com> wrote:
> >
> >
> > >4) We have cruise controls running classlibrary tests on DRLVM. We
> > >need to decide what will we do when DRLVM+Classlib cruise control
> > >reports failure.
> > >IMHO It's pretty obvious that before we have found a commit causing that
> > >we stop further commits.
> We can try and see how this works. It is important to reduce regressions,
> but depending on the time it takes to find the truant commit, repeated
> failures could literally stop new commits for a very long timetime. I think
> that 0 regression periods can exist in a project, but it cannot be zero
> regression always. However, I am perfectly OK with declaring a zero
> regression period for a trial period of a few weeks.
> >5) If the cause of failure is DRLVM commit than it's pretty obvious
> > >that we roll it back.
> Yes.
> >6) If the cause is Classlib test commit than we check wether the test
> > >is valid (e.g. run it on RI) and not VM-specific. If the test is valid
> > >we exclude it in DRLVM exclude list. If the test is invalid or
> > >VM-specific we roll it back or fix ASAP.
> Put it in the DRLVM exclude list and file a JIRA with the test case. Not
> sure what VM-specific means.  Ideally DRLVM and RI should be able to run
> the  same tests, so not being able to run it is a DRLVM problem, isn't it?
> >7) If the cause is a Classlib commit then this commit either reveals
> > >existing problem in DRLVM or introduces a regression. I don't have
> > >strong preferences here whether we'd better exclude the test or roll
> > >the commit back
> If the test fails only on DRLVM why should the classlib commit roll back? I
> would think that you may want to do the same as in 6). I think the catch is
> figuring out quickly what broke things...a DRLVM change, classlib change or
> test change. I think that in this post mortem model we will have to hope
> that this can be detected quickly, specially if commits are going to stop.
There is situations when test works OK on IBM VME and RI while fails
on DRLVM. And sometimes it could be the problem of test or classlib.
But I agree that in most cases it would be DRL VM problem.

SY, Alexey

> >8) We may also advise that committers don't commit changes just before
> > >going sleep or vacation. They should wait reasonable time to receive
> > >CC failure report if any. The problem is "reasonable time" is hardly
> > >predictable
> He should wait for results of the next CC run.

View raw message