harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Geir Magnusson Jr." <g...@pobox.com>
Subject Re: [general] aiming no regression
Date Fri, 01 Dec 2006 14:30:25 GMT


Mikhail Loenko wrote:
> seems like we need to refersh our agreements aiming better stability.
> There are questions to discuss and decision to be made.
> 
> 
> 1) We have exclude list for each paltform and for two VMs. We have
> discussed a separation of the common part but did not reach any
> agreement. To make fixing  VM- and platform-specific bugs easier I
> suggest that we agree to separate a
> common part.

I thought we agreed.  Either way, this is a side issue wrt regression.

> 
> 
> 2) Usually committers check the classlib patches on J9 and then
> commit. At this point checking the patches on DRLVM is not that
> convinient so at the moment we don't ask committers to do classlib
> pre-commit testing on DRLVM.

Yes, and we're getting near the time to switch that.  People have the 
freedom to do what they want, but we need to get this happening 
regularly.  I'll do something to make it easier for this to happen - 
I'll make a "drop-n-go" snapshot of DRLVM so it can be dropped directly 
into classlib's deploy/jdk/jre/bin as /default.

The only remaining issue that I can see is the infernal hythread.so 
problem.  Would we solve that simply by renaming our hythread.so in DRLVM?

> 
> 
> 3) CC should report each time state changes (pass to fail or fail to
> pass) and each
> time failure reason changed. Failure reason may change because some
> committers could miss failure notifiaction and do commit. Until we are
> able to say that failure reason has changed we report each failure and
> only the first success.

Sure.

> 
> 
> 4) We have cruise controls running classlibrary tests on DRLVM. We
> need to decide what will we do when DRLVM+Classlib cruise control
> reports failure.

Stop and fix the problem.  Is there really a question here?  I agree 
with Rana and Tim - we need to find a balance, and I think that with 
continued social awareness like this, we'll do better.  We can always 
revisit if things don't go well.

> IMHO It's pretty obvious that before we have found a commit causing that
> we stop further commits.
> 
> 
> 5) If the cause of failure is DRLVM commit than it's pretty obvious
> that we roll it back.
> 
> 
> 6) If the cause is Classlib test commit than we check wether the test
> is valid (e.g. run it on RI) and not VM-specific. If the test is valid
> we exclude it in DRLVM exclude list. If the test is invalid or
> VM-specific we roll it back or fix ASAP.

I don't completely agree, because it supports the idea that our tests 
must run on the RI, which can't be true for all classlib tests, and 
results in things like "isHarmony()".

We have two VMs to work with - one, a high-quality production grade VM 
(j9) and the other a soon-to-be high-quality production grade VM (DRLVM) :)

I suggest that a classlib failure is tested on both and then figure out 
what's the right answer, using the RI if possible.

> 
> 
> 7) If the cause is a Classlib commit then this commit either reveals
> existing problem in DRLVM or introduces a regression. I don't have
> strong preferences here whether we'd better exclude the test or roll
> the commit back

I think that it's fine to exclude the test *if* the commit identifies 
the problem and someone takes on the task of solving the problem 
immediately, in the interest of keeping things moving.

> 
> 
> 8) We may also advise that committers don't commit changes just before
> going sleep or vacation. They should wait reasonable time to receive
> CC failure report if any. The problem is "reasonable time" is hardly
> predictable
> 
> Comments?
> 
> Thanks,
> Mikhail

Mime
View raw message