couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dale Harvey <>
Subject Re: [DISCUSSION] Disallow all merges of PRs to master that cause tests to fail
Date Wed, 16 Aug 2017 07:43:36 GMT
+1, have found this useful / necessary with PouchDB over the years

Describing our workflow as it may be useful (its also very similiar to
mozillas). We have Tier 1
supported platform, node stable, stable releases of browsers etc a checkin
is not allowed to make
those go red, if something lands that makes them red it gets immediately

Tier 2 systems, like CouchDB 2.0 for the period it was under development.
We take a best effort
at keeping it green but it doesnt prevent patches from landing. If it goes
red it is someones job to get
it green, if it stays red for a long period of time it becomes unsupported.
(We are still in the process of moving
couch2 to tier1)

This allows us to stay pretty stable while not be too restrictive for
building new features / supporting new
platforms. The important thing however is the zero policy rule on Tier 1,
once there is any reason for
people to ignore failing tests they decay super quickly and it gets very
hard to bring them back, for a while
we had the policy of fixing things in master but there are always
complications, backing out, fixing the tests
and relanding is far more reliable

On 16 August 2017 at 07:46, Joan Touzet <> wrote:

> Hi committers,
> I'd like to propose a change to our policy on version control, namely
> that no check-ins be allowed on the master branch unless CI test runs
> against that PR are clean.
> We've worked hard as a group to get runs clean. We need to protect
> that achievement and investment in our test suite. That means not
> letting rogue check-ins slip by because we are ignoring a red X in
> GitHub (GH) from the Travis run.
> Things I see as exceptions:
> * Changes to things clearly not related to the test suite, i.e.
>   documentation, support scripts, rel/overlay/etc/ files, etc.
> * Changes already agreed upon in a previous PR/discussion for
>   administrative tasks
> Interesting situation right now for a discussion: Garren has a PR up[1]
> that enables the mango tests to be part of the standard Travis/Jenkins
> runs. Unfortunately, it doesn't pass on one of our platforms right now
> and that needs investigation. Should we allow the PR to land and fix
> the problems in master, or should the PR hold-up until it can land along
> with the fixes for the failing mango tests? I can see both sides of this
> argument.
> It may or may not be possible for our GH setup to actually prevent such
> checkins (the Apache GH setup is somewhat restricted, and various things
> like commit hooks and webhooks have to be configured by INFRA, not us).
> I'd like to further discuss whether people feel such a hook would be
> acceptable, onerous or otherwise. Personally, I worry that such a setup
> might prevent us from checking in some of the exceptions above, but if
> there is a way around it, we could proceed down that path.
> What do you think, sirs?[2]
> Joan
> [1]:
> [2]: It's a Mystery Science Theatre 3000 Joel reference. :)

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message