Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 3D558200B41 for ; Thu, 7 Jul 2016 22:40:40 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 3BDEA160A68; Thu, 7 Jul 2016 20:40:40 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 103BE160A4F for ; Thu, 7 Jul 2016 22:40:38 +0200 (CEST) Received: (qmail 26206 invoked by uid 500); 7 Jul 2016 20:40:37 -0000 Mailing-List: contact dev-help@spark.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list dev@spark.apache.org Received: (qmail 26195 invoked by uid 99); 7 Jul 2016 20:40:37 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 07 Jul 2016 20:40:37 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id D772D1A0753 for ; Thu, 7 Jul 2016 20:40:36 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -0.247 X-Spam-Level: X-Spam-Status: No, score=-0.247 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.426, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=yahoo.com Received: from mx2-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id hqFOyUoIUdpi for ; Thu, 7 Jul 2016 20:40:34 +0000 (UTC) Received: from nm10-vm0.bullet.mail.bf1.yahoo.com (nm10-vm0.bullet.mail.bf1.yahoo.com [98.139.213.147]) by mx2-lw-us.apache.org (ASF Mail Server at mx2-lw-us.apache.org) with ESMTPS id 389D75F2C2 for ; Thu, 7 Jul 2016 20:40:34 +0000 (UTC) Received: from [66.196.81.172] by nm10.bullet.mail.bf1.yahoo.com with NNFMP; 07 Jul 2016 20:40:27 -0000 Received: from [98.139.212.247] by tm18.bullet.mail.bf1.yahoo.com with NNFMP; 07 Jul 2016 20:40:27 -0000 Received: from [127.0.0.1] by omp1056.mail.bf1.yahoo.com with NNFMP; 07 Jul 2016 20:40:27 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 70424.2896.bm@omp1056.mail.bf1.yahoo.com X-YMail-OSG: TJnKE7wVM1ljuqxgbVRi20.QylalkV8gzqS61yLwVQSFEzWvvZYIgAI8zGwneAc 767cxDsLF6EvndS0O0bDNn.Nlg4nc7qAwbEd94uLBcy4Rm1bM736ZQfZeqttBBo0MlrPyN5K_jab ZMDv1UJXcebunPYLJzbojlq7LVZTyWgIRc41JXTakK8jE.9WPHniQXrCb3cIv3c6kBJ44n4YfKpa OPgU8aqqzDfqmGhPMUsEJ6AW_pEICaR52dIq6mYXiD1crgyTjW2h8RnQ6afKP0EK9N7oSevpU5kK jkgRNErEJugQ40JVI9hITdfAKIIAlOYzZU3L8pPEtPVz5A2wis4BDiiLtR.ACaL1GrLnv6bA7alm SlpQDAQD1X5ANllgKC_y3bpVh32Yvmpgi8yXJUvin8nh5KS3VFqnGCLTORUFS.7bqcddEbviDD79 CGgNugm6LP0aDU0WJtaoi4_740SZuDkydsx5ZOKEFqlfQnhPO5vRNBLSdxuMP5x00bx.4U6Xmq6P EMl4w37SuL7Lpzx3r3AWMfw7Q.urUQg-- Received: from jws10685.mail.bf1.yahoo.com by sendmailws139.mail.bf1.yahoo.com; Thu, 07 Jul 2016 20:40:26 +0000; 1467924026.616 Date: Thu, 7 Jul 2016 20:40:26 +0000 (UTC) From: Tom Graves Reply-To: Tom Graves To: Sean Owen Cc: "dev@spark.apache.org" Message-ID: <1673838343.3496708.1467924026179.JavaMail.yahoo@mail.yahoo.com> In-Reply-To: References: <1900450842.3602015.1467917252211.JavaMail.yahoo@mail.yahoo.com> Subject: Re: [DISCUSS] Minimize use of MINOR, BUILD, and HOTFIX w/ no JIRA MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_3496707_156232634.1467924026171" archived-at: Thu, 07 Jul 2016 20:40:40 -0000 ------=_Part_3496707_156232634.1467924026171 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable I think the problems comes in with your definition as well as peoples inter= pretation of that. =C2=A0I don't agree with your statement of "where the "h= ow" is different from the "what"". =C2=A0 This could apply to a lot of things. =C2=A0I could easily file a jira that = says remove synchronization on routine x, then change a lock. =C2=A0No disc= ussion needed and the how is the same as the what. =C2=A0But it could have = huge impact on the code and definitely should have a jira. =C2=A0It may be = a contrived example but there is a lot of leeway in that. =C2=A0Which is wh= y I think Patrick originally sent this email and you said it yourself above= that some of the things reverted =C2=A0weren't trivial enough to begin wit= h. =C2=A0That just proves people can't make that judgement call themselves.= =C2=A0So why not just file a jira for everything? =C2=A0Another example of= this could be doc changes. =C2=A0you may think they are trivial but if som= eone changes the docs and remove configs or change the wording such that us= ers don't understand, then to me it should have had a jira and possibly dis= cussion before changing. =C2=A0 So based on that it seems like spending the 5 to 30 seconds to file a jira = would only help in tracking things and isn't much overhead. =C2=A0 We also base our release notes and other things on jira.=C2=A0 Also for hotfixes I think they should have the original jira or a separate = jira (with brokenby linked to original), again for tracking purposes. If we= check something into master and then later want to cherry-pick it back, I = might just pick the original commit and totally miss this "HOTFIX" that was= required if they aren't properly linked. Tom=20 On Thursday, July 7, 2016 2:56 PM, Sean Owen wrote= : =20 I don't agree that every change needs a JIRA, myself. Really, we didn't choose to have this system split across JIRA and Github PRs. It's necessitated by how the ASF works (and with some good reasons). But while we have this dual system, I figure, let's try to make some sense of it. I think it makes sense to make a JIRA for any non-trivial change. What's non-trivial? where the "how" is different from the "what". That is, if the JIRA is not just a repeat of the pull request, they should probably be separate. But, if the change is so simple that describing it amounts to dictating how it's implemented -- well, seems like a JIRA is just overhead. ONe problem that I think happened above was: pretty non-trivial things were being merged without a JIRA. The evidence? they were reverted. That means their effect was not quite obvious. They probably deserved more discussion. Anything that needs some discussion probably deserves a JIRA. Also: we have some hot-fixes here that aren't connected to JIRAs. Either they belong with an existing JIRA and aren't tagged correctly, or, again, are patching changes that weren't really trivial enough to skip a JIRA to begin with. On Thu, Jul 7, 2016 at 7:47 PM, Tom Graves w= rote: > Popping this back up to the dev list again.=C2=A0 I see a bunch of checki= ns with > minor or hotfix. > > It seems to me we shouldn't be doing this, but I would like to hear thoug= hts > from others.=C2=A0 I see no reason we can't have a jira for each of those= issues, > it only takes a few seconds to file one and it makes things much easier t= o > track. > > For instance, I tend to watch the jiras on the mailing list and if I hit = an > issue I search jira to see if there is existing one for it, but if there > isn't jira then I don't see and can't find what someone perhaps already > fixed with a [MINOR] checkin. > > Tom > > > On Saturday, June 6, 2015 11:02 AM, Patrick Wendell > wrote: > > > Hey All, > > Just a request here - it would be great if people could create JIRA's > for any and all merged pull requests. The reason is that when patches > get reverted due to build breaks or other issues, it is very difficult > to keep track of what is going on if there is no JIRA. Here is a list > of 5 patches we had to revert recently that didn't include a JIRA: > >=C2=A0 =C2=A0 Revert "[MINOR] [BUILD] Use custom temp directory during bui= ld." >=C2=A0 =C2=A0 Revert "[SQL] [TEST] [MINOR] Uses a temporary log4j.properti= es in > HiveThriftServer2Test to ensure expected logging behavior" >=C2=A0 =C2=A0 Revert "[BUILD] Always run SQL tests in master build." >=C2=A0 =C2=A0 Revert "[MINOR] [CORE] Warn users who try to cache RDDs with > dynamic allocation on." >=C2=A0 =C2=A0 Revert "[HOT FIX] [YARN] Check whether `/lib` exists before > listing its files" > > The cost overhead of creating a JIRA relative to other aspects of > development is very small. If it's *really* a documentation change or > something small, that's okay. > > But anything affecting the build, packaging, etc. These all need to > have a JIRA to ensure that follow-up can be well communicated to all > Spark developers. > > Hopefully this is something everyone can get behind, but opened a > discussion here in case others feel differently. > > - Patrick > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org > For additional commands, e-mail: dev-help@spark.apache.org > > > ------=_Part_3496707_156232634.1467924026171 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
I think the problems comes in with your definition as well as peoples = interpretation of that.  I don't agree with your statement of "where the "how" is different from the "what"".  = ;

This could apply to a lot= of things.  I could easily file a jira that says remove synchronizati= on on routine x, then change a lock.  No discussion needed and the how= is the same as the what.  But it could have huge impact on the code a= nd definitely should have a jira.  
It may be a contrived example but there is a lot of leeway in tha= t.  Which is why I think Patrick originally sent this email and you sa= id it yourself above that some of the things reverted  weren't trivial= enough to begin with.  That just proves people can't make that judgem= ent call themselves.  So why not just file a jira for everything? &nbs= p;Another example of this could be doc changes.  you may think they ar= e trivial but if someone changes the docs and remove configs or change the = wording such that users don't understand, then to me it should have had a j= ira and possibly discussion before changing.  

So = based on that it seems like spending the 5 to 30 seconds to file a jira wou= ld only help in tracking things and isn't much overhead.  

We also base our release notes and other th= ings on jira. 

Also for hotfixes I think they shou= ld have the original jira or a separate jira (with brokenby linked to origi= nal), again for tracking purposes. If we check something into master and th= en later want to cherry-pick it back, I might just pick the original commit= and totally miss this "HOTFIX" that was required if they aren't properly l= inked.

Tom


<= div class=3D"yiv2106211103yqt7151164001" id=3D"yiv2106211103yqt19247">
=
On Thursday, July 7, 2016= 2:56 PM, Sean Owen <sowen@cloudera.com> wrote:


I don't agree that every change needs a JIRA, myself. R= eally, we
didn't choose to have this system split across = JIRA and Github PRs.
It's necessitated by how the ASF wor= ks (and with some good reasons).
But while we have this d= ual system, I figure, let's try to make some
sense of it.=

I think it makes sense to make a JIRA= for any non-trivial change.
What's non-trivial? where th= e "how" is different from the "what". That
is, if the JIR= A is not just a repeat of the pull request, they should
p= robably be separate. But, if the change is so simple that describing
it amounts to dictating how it's implemented -- well, seems lik= e a
JIRA is just overhead.

ONe problem that I think happened above was: pretty non-trivial thin= gs
were being merged without a JIRA. The evidence? they w= ere reverted.
That means their effect was not quite obvio= us. They probably deserved
more discussion. Anything that= needs some discussion probably deserves
a JIRA.

Also: we have some hot-fixes here that aren't = connected to JIRAs.
Either they belong with an existing J= IRA and aren't tagged correctly,
or, again, are patching = changes that weren't really trivial enough to
skip a JIRA= to begin with.

On Thu, Jul 7, 2016 at 7= :47 PM, Tom Graves <tgraves_cs@yahoo.com.invalid> wrote:
> Popping this back up to the dev list again.  I see a bunch = of checkins with
> minor or hotfix.
= >
> It seems to me we shouldn't be doing this, but = I would like to hear thoughts
> from others.  I s= ee no reason we can't have a jira for each of those issues,
> it only takes a few seconds to file one and it makes things much ea= sier to
> track.
>
> For instance, I tend to watch the jiras on the mailing list and if= I hit an
> issue I search jira to see if there is exi= sting one for it, but if there
> isn't jira then I don= 't see and can't find what someone perhaps already
> f= ixed with a [MINOR] checkin.
>
> = Tom
>
>
> On= Saturday, June 6, 2015 11:02 AM, Patrick Wendell <pwendell@gmail.com>
> wrote:
>
>
> Hey All,
>
> Just a reques= t here - it would be great if people could create JIRA's
= > for any and all merged pull requests. The reason is that when patches<= br clear=3D"none">> get reverted due to build breaks or other issues, it= is very difficult
> to keep track of what is going on= if there is no JIRA. Here is a list
> of 5 patches we= had to revert recently that didn't include a JIRA:
><= br clear=3D"none">>    Revert "[MINOR] [BUILD] Use custom tem= p directory during build."
>    Revert "[SQ= L] [TEST] [MINOR] Uses a temporary log4j.properties in
&g= t; HiveThriftServer2Test to ensure expected logging behavior"
>    Revert "[BUILD] Always run SQL tests in master buil= d."
>    Revert "[MINOR] [CORE] Warn users = who try to cache RDDs with
> dynamic allocation on.">    Revert "[HOT FIX] [YARN] Check whether = `/lib` exists before
> listing its files"
>
> The cost overhead of creating a JIRA rela= tive to other aspects of
> development is very small. = If it's *really* a documentation change or
> something= small, that's okay.
>
> But anyt= hing affecting the build, packaging, etc. These all need to
> have a JIRA to ensure that follow-up can be well communicated to al= l
> Spark developers.
>
> Hopefully this is something everyone can get behind, but ope= ned a
> discussion here in case others feel differentl= y.
>
> - Patrick
>
> ---------------------------------------------= ------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> For additional= commands, e-mail: dev-help@spark.apache.org
>
>
>


------=_Part_3496707_156232634.1467924026171--