spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tom Graves <>
Subject Re: [DISCUSS] Minimize use of MINOR, BUILD, and HOTFIX w/ no JIRA
Date Thu, 07 Jul 2016 20:40:26 GMT
I think the problems comes in with your definition as well as peoples interpretation of that.
 I don't agree with your statement of "where the "how" is different from the "what"".  
This could apply to a lot of things.  I could easily file a jira that says remove synchronization
on routine x, then change a lock.  No discussion needed and the how is the same as the what.
 But it could have huge impact on the code and definitely should have a jira.  It may be
a contrived example but there is a lot of leeway in that.  Which is why I think Patrick originally
sent this email and you said it yourself above that some of the things reverted  weren't
trivial enough to begin with.  That just proves people can't make that judgement call themselves.
 So why not just file a jira for everything?  Another example of this could be doc changes.
 you may think they are trivial but if someone changes the docs and remove configs or change
the wording such that users don't understand, then to me it should have had a jira and possibly
discussion before changing.  
So based on that it seems like spending the 5 to 30 seconds to file a jira would only help
in tracking things and isn't much overhead.  
We also base our release notes and other things on jira. 
Also for hotfixes I think they should have the original jira or a separate jira (with brokenby
linked to original), again for tracking purposes. If we check something into master and then
later want to cherry-pick it back, I might just pick the original commit and totally miss
this "HOTFIX" that was required if they aren't properly linked.

    On Thursday, July 7, 2016 2:56 PM, Sean Owen <> wrote:

 I don't agree that every change needs a JIRA, myself. Really, we
didn't choose to have this system split across JIRA and Github PRs.
It's necessitated by how the ASF works (and with some good reasons).
But while we have this dual system, I figure, let's try to make some
sense of it.

I think it makes sense to make a JIRA for any non-trivial change.
What's non-trivial? where the "how" is different from the "what". That
is, if the JIRA is not just a repeat of the pull request, they should
probably be separate. But, if the change is so simple that describing
it amounts to dictating how it's implemented -- well, seems like a
JIRA is just overhead.

ONe problem that I think happened above was: pretty non-trivial things
were being merged without a JIRA. The evidence? they were reverted.
That means their effect was not quite obvious. They probably deserved
more discussion. Anything that needs some discussion probably deserves

Also: we have some hot-fixes here that aren't connected to JIRAs.
Either they belong with an existing JIRA and aren't tagged correctly,
or, again, are patching changes that weren't really trivial enough to
skip a JIRA to begin with.

On Thu, Jul 7, 2016 at 7:47 PM, Tom Graves <> wrote:
> Popping this back up to the dev list again.  I see a bunch of checkins with
> minor or hotfix.
> It seems to me we shouldn't be doing this, but I would like to hear thoughts
> from others.  I see no reason we can't have a jira for each of those issues,
> it only takes a few seconds to file one and it makes things much easier to
> track.
> For instance, I tend to watch the jiras on the mailing list and if I hit an
> issue I search jira to see if there is existing one for it, but if there
> isn't jira then I don't see and can't find what someone perhaps already
> fixed with a [MINOR] checkin.
> Tom
> On Saturday, June 6, 2015 11:02 AM, Patrick Wendell <>
> wrote:
> Hey All,
> Just a request here - it would be great if people could create JIRA's
> for any and all merged pull requests. The reason is that when patches
> get reverted due to build breaks or other issues, it is very difficult
> to keep track of what is going on if there is no JIRA. Here is a list
> of 5 patches we had to revert recently that didn't include a JIRA:
>    Revert "[MINOR] [BUILD] Use custom temp directory during build."
>    Revert "[SQL] [TEST] [MINOR] Uses a temporary in
> HiveThriftServer2Test to ensure expected logging behavior"
>    Revert "[BUILD] Always run SQL tests in master build."
>    Revert "[MINOR] [CORE] Warn users who try to cache RDDs with
> dynamic allocation on."
>    Revert "[HOT FIX] [YARN] Check whether `/lib` exists before
> listing its files"
> The cost overhead of creating a JIRA relative to other aspects of
> development is very small. If it's *really* a documentation change or
> something small, that's okay.
> But anything affecting the build, packaging, etc. These all need to
> have a JIRA to ensure that follow-up can be well communicated to all
> Spark developers.
> Hopefully this is something everyone can get behind, but opened a
> discussion here in case others feel differently.
> - Patrick
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

View raw message