harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Ellison <t.p.elli...@gmail.com>
Subject Re: [general] Discussion: how to keep up stability and fast progress all together?
Date Mon, 02 Apr 2007 17:07:22 GMT
Stepan Mishura wrote:
> Sure this approach makes sense and I think we should accept and follow
> it. I see only one issue here - it lets instabilities get accumulated
> and present unnoticed (ignored?) in the code for some period of time.
> This may result that minor update can have unintended consequences.

Apologies for not being so clear, we should continue to ensure that the
automated tests and pre-commit tests are run as we do today to ensure
day-to-day stability.  Of course, as work progresses there may be
regressions that require a commit to be backed-out (or better still, fixed).

The stability period approaching a Milestone is a pact that only
stability enhancing fixes are applied to ensure there is no last minute
disruption during the enhanced testing and declaration of the release.

> Currently if we identify a regression we try to find a guilty commit
> and to fix it or do rollback. I think it is the right way – we keep
> code base in a good shape and don't let a number of known problems to
> grow. This approach showed its efficiency and the only thing I can do
> here is only encourage all contributors to run all available tests
> after doing any non-trivial change.


> But it seems for intermittent
> failures the approach with running all testing scenarios doesn't work
> well – usually they are not immediately detected. And it's hard to
> find guilty update after a long time so we tend to put such tests to
> exclude list.

Right, and we should triage these and draw extra focus on them during
the stability pass.

> I'd like to propose the next approach that may help us to know about
> instabilities: develop (or take existing one, for example, Eclipse
> hello world) a scenario for testing stability and configure CC to run
> it at all times. The stability scenario must be the only one scenario
> for CC; it must be short (no longer then an hour), test JRE in stress
> conditions and cover most of functionality. If the scenario fails then
> all newly committed updates are subject for investigation and fix (or
> rollback).

None from me.


View raw message