incubator-cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcus Sorensen <shadow...@gmail.com>
Subject Re: How we treat broken builds.
Date Mon, 18 Feb 2013 02:03:48 GMT
Are there jenkins reports the committers can subscribe to and get email
when there are breakages? I know at least once 4.1 was broken while I was
in the process of testing a commit (just a quick sanity check I do to avoid
being the guy who causes these sorts of things), and then I committed on
top of that broken branch. If there are such emails (and I suspect there
are), it would help me to know when to stop working on the branch, and all
of the committers should probably sign up.
On Feb 17, 2013 6:44 PM, "David Nalley" <david@gnsa.us> wrote:

Hi folks,

I know folks have said this elsewhere, but I'd like to reiterate it.

I am somewhat frustrated with our reaction to broken builds. In
general it seems we don't care, and this makes it more difficult to
fix problems. Jenkins reporting a broken build (be it a broken run of
RAT, failure to compile, failure of a unit test, building docs, etc.)
should be our Andon cord [1]. We should all stop commits that aren't
fixing the broken build. To illustrate why this is a problem,

RAT failures started occurring recently, this keeps us from testing
whether CloudStack builds, because each build is conditioned on the
successful completion of the test before it. That in turn keeps
apidocs from building,  which keeps marvin from building, which keeps
documentation from building. We essentially are blind until it gets
fixed.

That means, that like TPS, when the andon cord gets pulled we all need
to focus on the problem, and not continuing our own work and ignoring
the problem and potentially contributing to making the solution more
painful.

My requests:

If you see that the branch you are working on is broken - please don't
commit to it, unless your commit is going to fix it.
If you see that the branch you are working on is broken - please help
fix it. (This should become priority #1, for everyone)
Please don't make a commit the last thing you do before going offline
- make sure that your commit isn't breaking any of the tests before
you leave. If your commit breaks something and you've gone offline for
16 hours, you've made life painful for others, so make sure there is
some buffer between your last commit and you going offline so you can
see and remedy any problems that arise.

And just for the record - as of this moment, 4.1 and master are broken
at various stages.


--David

[1] http://en.wikipedia.org/wiki/Andon_(manufacturing)

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message