spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Wendell <>
Subject Re: When to cut RCs
Date Thu, 03 Dec 2015 03:07:45 GMT
In terms of advertising to people the status of the release and whether an
RC is likely to go out, the best mechanism I can think of is our current
mechanism of using JIRA and respecting the semantics of a blocker JIRA. We
could do a better job though creating a JIRA dashboard for each release and
linking to it publicly so it's very clear to people what is going on. I
have always used one privately when managing previous releases, but no
reason we can't put one up on the website or wiki.

IMO a mailing list is not a great mechanism for the fine-grained work of
release management because of the sheer complexity and volume of finalizing
a spark release. Being a release manager means tracking over a course of
several weeks typically dozens of distinct issues and trying to prioritize
them, get more clarity from the report of those issues, possibly reaching
out to people on the phone or in person to get more details, etc. You want
a mutable dashboard where you can convey the current status clearly.

What might be good in the early stages is a weekly e-mail to the dev@ list
just refreshing what is on the JIRA and letting people know how things are
looking. So someone just passing by has some idea of how things are going
and can chime in, etc.

Once an RC is cut then we do mostly rely on the mailing list for
discussion. At that point the number of known issues is small enough I
think to discuss in an all-to-all fashion.

- Patrick

On Wed, Dec 2, 2015 at 1:25 PM, Sean Owen <> wrote:

> On Wed, Dec 2, 2015 at 9:06 PM, Michael Armbrust <>
> wrote:
> > This can be debated, but I explicitly ignored test and documentation
> issues.
> > Since the docs are published separately and easy to update, I don't think
> > its worth further disturbing the release cadence for these JIRAs.
> It makes sense to not hold up an RC since they don't affect testing of
> functionality. Prior releases have ultimately gone out with doc issues
> still outstanding (and bugs) though. This doesn't seem to be on
> anyone's release checklist, and maybe part of it is because they're
> let slide for RCs.  Your suggestion to check-point release status
> below sounds spot on; I sort of tried to do that earlier.
> > Up until today various committers have told me that there were known
> issues
> > with branch-1.6 that would cause them to -1 the release.  Whenever this
> > happened, I asked them to ensure there was a properly targeted blocker
> > open so people could publicly track the status of the release.  As long
> as
> > such issues were open, I only published a preview since making an RC is
> > pretty high cost.
> Makes sense if these are all getting translated into Blockers and
> resolved before an RC. It's the simplest mechanism to communicate and
> track this in a distributed way.
> "No blockers" is a minimal criterion for release. It still seems funny
> to release with so many issues targeted for 1.6.0, including issues
> that aren't critical or bugs. Sure, that's just hygiene. But without
> it, do people take "Target Version" seriously? if they don't, is there
> any force guiding people to prioritize or decide what to (not) work
> on? I'm sure the communication happens, just doesn't seem like it's
> fully on JIRA, which is ultimately suboptimal.
> > I actually did spent quite a bit of time asking people to close various
> > umbrella issues, and I was pretty strict about watching JIRA throughout
> the
> > process.  Perhaps as an additional step, future preview releases or
> branch
> > cuts can include a link to an authoritative dashboard that we will use to
> > decide when we are ready to make an RC.  I'm also open to other
> suggestions.
> Yes, that's great. It takes the same effort from everyone. Having a
> green light on a dashboard at release time is only the symptom of
> decent planning. The effect I think it really needs to have occurs
> now: what's really probably on the menu for 1.7? and periodically
> track against that goal. Then the release process is with any luck
> just a formality with no surprises.
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

View raw message