harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rana Dasgupta" <rdasg...@gmail.com>
Subject Re: [general] aiming no regression
Date Fri, 01 Dec 2006 05:57:55 GMT
Mikhail,
   Please see below...


On 11/30/06, Mikhail Loenko <mloenko@gmail.com> wrote:
>
>
> >4) We have cruise controls running classlibrary tests on DRLVM. We
> >need to decide what will we do when DRLVM+Classlib cruise control
> >reports failure.
> >IMHO It's pretty obvious that before we have found a commit causing that
> >we stop further commits.


We can try and see how this works. It is important to reduce regressions,
but depending on the time it takes to find the truant commit, repeated
failures could literally stop new commits for a very long timetime. I think
that 0 regression periods can exist in a project, but it cannot be zero
regression always. However, I am perfectly OK with declaring a zero
regression period for a trial period of a few weeks.

>5) If the cause of failure is DRLVM commit than it's pretty obvious
> >that we roll it back.


Yes.

>6) If the cause is Classlib test commit than we check wether the test
> >is valid (e.g. run it on RI) and not VM-specific. If the test is valid
> >we exclude it in DRLVM exclude list. If the test is invalid or
> >VM-specific we roll it back or fix ASAP.


Put it in the DRLVM exclude list and file a JIRA with the test case. Not
sure what VM-specific means.  Ideally DRLVM and RI should be able to run
the  same tests, so not being able to run it is a DRLVM problem, isn't it?

>7) If the cause is a Classlib commit then this commit either reveals
> >existing problem in DRLVM or introduces a regression. I don't have
> >strong preferences here whether we'd better exclude the test or roll
> >the commit back


If the test fails only on DRLVM why should the classlib commit roll back? I
would think that you may want to do the same as in 6). I think the catch is
figuring out quickly what broke things...a DRLVM change, classlib change or
test change. I think that in this post mortem model we will have to hope
that this can be detected quickly, specially if commits are going to stop.

>8) We may also advise that committers don't commit changes just before
> >going sleep or vacation. They should wait reasonable time to receive
> >CC failure report if any. The problem is "reasonable time" is hardly
> >predictable


He should wait for results of the next CC run.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message