harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mikhail Loenko" <mloe...@gmail.com>
Subject Re: [general] Discussion: how to keep up stability and fast progress all together?
Date Tue, 03 Apr 2007 02:19:06 GMT
2007/4/3, Tim Ellison <t.p.ellison@gmail.com>:
> Stepan Mishura wrote:
> > Sure this approach makes sense and I think we should accept and follow
> > it. I see only one issue here - it lets instabilities get accumulated
> > and present unnoticed (ignored?) in the code for some period of time.
> > This may result that minor update can have unintended consequences.
> Apologies for not being so clear, we should continue to ensure that the
> automated tests and pre-commit tests are run as we do today to ensure
> day-to-day stability.  Of course, as work progresses there may be
> regressions that require a commit to be backed-out (or better still, fixed).
> The stability period approaching a Milestone is a pact that only
> stability enhancing fixes are applied to ensure there is no last minute
> disruption during the enhanced testing and declaration of the release.

I agree with "feature freeze" periods. An open question is what is the frequency
of milestones and duration of "feature freeze" periods before them.
I think we should also have code freeze periods: when we do last-minute testing
to validate the milestone candidate build.


> > Currently if we identify a regression we try to find a guilty commit
> > and to fix it or do rollback. I think it is the right way – we keep
> > code base in a good shape and don't let a number of known problems to
> > grow. This approach showed its efficiency and the only thing I can do
> > here is only encourage all contributors to run all available tests
> > after doing any non-trivial change.
> Agreed.
> > But it seems for intermittent
> > failures the approach with running all testing scenarios doesn't work
> > well – usually they are not immediately detected. And it's hard to
> > find guilty update after a long time so we tend to put such tests to
> > exclude list.
> Right, and we should triage these and draw extra focus on them during
> the stability pass.
> > I'd like to propose the next approach that may help us to know about
> > instabilities: develop (or take existing one, for example, Eclipse
> > hello world) a scenario for testing stability and configure CC to run
> > it at all times. The stability scenario must be the only one scenario
> > for CC; it must be short (no longer then an hour), test JRE in stress
> > conditions and cover most of functionality. If the scenario fails then
> > all newly committed updates are subject for investigation and fix (or
> > rollback).
> None from me.
> Regards,
> Tim

View raw message