spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Wendell <pwend...@gmail.com>
Subject Re: Signal/Noise Ratio
Date Sat, 22 Feb 2014 20:35:58 GMT
Hey Chris,

Would the following be consistent with the Apache guidelines?

(a) We establish a culture of not having overall design discussions on
github. Design discussions should to occur on JIRA or on the dev list.
IMO this is pretty much already true, but there are a few exceptions.
(b) We add a mailing list called github@s.a.o which receives the
github traffic. This way everything is available in Apache infra.
(c) Because of our use of JIRA it might make sense to have an
issues@s.a.o list as well similar to what YARN and other projects use.

The github chatter is so noisy that I think, overall, it decreases
engagement with the official developer list. This is the opposite of
what we want.

- Patrick

On Sat, Feb 22, 2014 at 11:34 AM, Mattmann, Chris A (3980)
<chris.a.mattmann@jpl.nasa.gov> wrote:
> Hi Everyone,
>
> The biggest thing is simply making sure that the dev@<project>a.o list is
> meaningful
> and that meaningful development isn't going on elsewhere that constitute
> "decisions" for the Apache project as reified in code contributions and
> overall
> stewardship of the effort.
>
> I noticed in a few emails from Github relating to comments on Github Pull
> Requests
> some conversation which I deemed to be relevant to the project, so I
> brought this
> up and it came up during graduation.
>
> Here's a general rule of thumb: it's fine if devs converse e.g., on
> Github, etc.,
> and even if it's project discussion *so long as* that relevant project
> discussion
> makes it way in some form to the actual, bona fide project's
> "dev@<project>a.o list",
> giving others in the community not necessarily on Github or watching
> Github or part
> of that non Apache conversation to comment, and be part of the community
> led decisions
> for the project there.
>
> Making its way to that bona fide Apache project dev list can happen in
> several ways.
>
> 1. by simply direct 1:1 mapping from Github comments which I see Apache
> project
> related dev discussion on from time to time and believe fits the criteria
> I'm describing
> above to the project's dev@<project>.a.o list.
>
> 2. by not 1:1 mapping all Github conversation to the dev@<project>.a.o
> list, but to
> some other list, e.g., github@<project>a.o, for example (or any of the
> others being
> discussed) *so long as*, and this is key, that those discussions on Github
> get summarized
> on the dev@<project>.a.o list giving everyone an opportunity to
> participate in the development
> by being *here at Apache*.
>
> 3. By not worrying about Github at all and simply doing all the
> development here at
> the ASF.
>
> 4. Others..
>
> My feeling is that some combination of #1 and #2 can pass muster, and the
> Apache Spark
> community can decide. That said, noise reduction can also lead to loss of
> precision and
> accuracy and don't be surprised in reducing that noise if some key thing
> makes it onto
> a Github PR but didn't make it onto the dev list b/c we are all human and
> forgot to summarize
> it there. Even if that happens, we assume everyone has good intentions and
> we simply
> address those issues when/if they come up.
>
> Cheers,
> Chris
>
>
>
>
> -----Original Message-----
> From: Sandy Ryza <sandy.ryza@cloudera.com>
> Reply-To: "dev@spark.incubator.apache.org" <dev@spark.incubator.apache.org>
> Date: Saturday, February 22, 2014 11:19 AM
> To: "dev@spark.incubator.apache.org" <dev@spark.incubator.apache.org>
> Subject: Re: Signal/Noise Ratio
>
>>Hadoop subprojects (MR, YARN, HDFS) each have a "dev" list that contains
>>discussion as well as a single email whenever a JIRA is filed, and an
>>"issues" list with all the JIRA activity.  I think this works out pretty
>>well.  Subscribing just to the dev list, I can keep up with changes that
>>are going to be made and follow the ones I care about.  And the issues
>>list
>>is there if I want the firehose.
>>
>>Is Apache actually prescriptive that a list with "dev" in its name needs
>>to
>>contain all discussion?  If so, most projects I've followed are violating
>>this.
>>
>>
>>On Fri, Feb 21, 2014 at 7:54 PM, Kay Ousterhout
>><keo@eecs.berkeley.edu>wrote:
>>
>>> It looks like there's at least one other apache project, jclouds, that
>>> sends the github notifications to a separate notifications@ list (see
>>>
>>>
>>>http://mail-archives.apache.org/mod_mbox/incubator-general/201402.mbox/%3
>>>C1391721862.67613.YahooMailNeo%40web172602.mail.ir2.yahoo.com%3E
>>> ).
>>>  Given that many people are annoyed by getting the messages on this
>>>list,
>>> and that there is some precedent for sending them to a different list,
>>>I'd
>>> be in favor of doing that.
>>>
>>>
>>> On Fri, Feb 21, 2014 at 6:18 PM, Mattmann, Chris A (3980) <
>>> chris.a.mattmann@jpl.nasa.gov> wrote:
>>>
>>> > Sweet great job Reynold.
>>> >
>>> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>> > Chris Mattmann, Ph.D.
>>> > Chief Architect
>>> > Instrument Software and Science Data Systems Section (398)
>>> > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>>> > Office: 171-283, Mailstop: 171-246
>>> > Email: chris.a.mattmann@nasa.gov
>>> > WWW:  http://sunset.usc.edu/~mattmann/
>>> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>> > Adjunct Associate Professor, Computer Science Department
>>> > University of Southern California, Los Angeles, CA 90089 USA
>>> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > -----Original Message-----
>>> > From: Reynold Xin <rxin@databricks.com>
>>> > Reply-To: "dev@spark.incubator.apache.org" <
>>> dev@spark.incubator.apache.org
>>> > >
>>> > Date: Friday, February 21, 2014 6:08 PM
>>> > To: "dev@spark.incubator.apache.org" <dev@spark.incubator.apache.org>
>>> > Subject: Re: Signal/Noise Ratio
>>> >
>>> > >FYI I submitted an ASF INFRA ticket on granting the AMPLab Jenkins
>>> > >permission to use the github commit status API.
>>> > >
>>> > >If that goes through, we can configure Jenkins to use the commit
>>>status
>>> > >API
>>> > >without leaving comments on the pull requests.
>>> > >
>>> > >https://issues.apache.org/jira/browse/INFRA-7367
>>> > >
>>> > >
>>> > >
>>> > >On Fri, Feb 21, 2014 at 11:14 AM, Ethan Jewett <esjewett@gmail.com>
>>> > wrote:
>>> > >
>>> > >> Thanks for the pointer Aaron. Very helpful.
>>> > >>
>>> > >> I won't harp on this any more after this email: my reading is that
>>>the
>>> > >>main
>>> > >> concern is archiving discussion, which could be achieved using
a
>>> > >>separate
>>> > >> mailing list. Major decisions should clearly happen on the dev
>>>list so
>>> > >> everyone is informed, but I don't see a situation where that hadn't
>>> been
>>> > >> happening anyway (which is why I read the dev list regularly,
>>> sometimes
>>> > >> look at the archives, and am struggling with the Github messages
>>>and
>>> > >> pitying those not using Gmail filters).
>>> > >>
>>> > >>
>>> > >> On Fri, Feb 21, 2014 at 12:51 PM, Aaron Davidson
>>><ilikerps@gmail.com>
>>> > >> wrote:
>>> > >>
>>> > >> > I don't have an official policy to point you to, but Chris
>>>Mattmann
>>> > >>(our
>>> > >> > Apache project mentor) summarized some of the points in this
>>>thread,
>>> > >>and
>>> > >> > here is the original concern that caused us to make this change:
>>> > >> >
>>> > >> >
>>> > >> >
>>> > >>
>>> > >>
>>> >
>>>
>>>http://mail-archives.apache.org/mod_mbox/incubator-general/201402.mbox/%3
>>> >
>>>>>CCAAS6=7hkCiT093nXVMcUus8Z-5XCDn=cQ5trjN_Kz9ARe9H=RA@mail.gmail.com%3E
>>> > >> >
>>> > >> >
>>> > >> > On Fri, Feb 21, 2014 at 8:08 AM, Ethan Jewett
>>><esjewett@gmail.com>
>>> > >> wrote:
>>> > >> >
>>> > >> > > Or not off-list. Sorry folks :-) Anyone should feel free
to
>>> educate
>>> > >>me
>>> > >> > > either on the policy or on mailing list use ;-)
>>> > >> > >
>>> > >> > > On Friday, February 21, 2014, Ethan Jewett <esjewett@gmail.com>
>>> > >>wrote:
>>> > >> > >
>>> > >> > > > Hi Aaron,
>>> > >> > > >
>>> > >> > > > Off-list message here. Can you point me to this
policy? Due
>>>to
>>> > >>some
>>> > >> > > > previous experiences here, I'm under the impression
that it
>>> > >>doesn't
>>> > >> > > exist.
>>> > >> > > > I can't find it on the Apache website.
>>> > >> > > >
>>> > >> > > > Thanks,
>>> > >> > > > Ethan
>>> > >> > > >
>>> > >> > > > On Tuesday, February 18, 2014, Aaron Davidson <
>>> ilikerps@gmail.com
>>> > >> > > <javascript:_e(%7B%7D,'cvml','ilikerps@gmail.com');>>
>>> > >> > > > wrote:
>>> > >> > > >
>>> > >> > > >> This is due, unfortunately, to Apache policies
that all
>>> > >> > > >> development-related
>>> > >> > > >> discussion should take place on the dev list.
As we are
>>> > >>attempting
>>> > >> to
>>> > >> > > >> graduate from an incubating project to an Apache
top level
>>> > >>project,
>>> > >> > > there
>>> > >> > > >> were some concerns raised about GitHub, and
the fastest
>>> solution
>>> > >>to
>>> > >> > > avoid
>>> > >> > > >> conflict related to our graduation was to CC
dev@ for all
>>> GitHub
>>> > >> > > >> messages.
>>> > >> > > >> Once our graduation is complete, we may be able
to find a
>>>less
>>> > >>noisy
>>> > >> > way
>>> > >> > > >> of
>>> > >> > > >> dealing with these messages.
>>> > >> > > >>
>>> > >> > > >> In the meantime, one simple solution is to filter
out all
>>> > >>messages
>>> > >> > that
>>> > >> > > >> come from git@git.apache.org and are destined
to
>>> > >> > > >> dev@spark.incubator.apache.org.
>>> > >> > > >>
>>> > >> > > >>
>>> > >> > > >> On Tue, Feb 18, 2014 at 10:04 AM, Gerard Maas
<
>>> > >> gerard.maas@gmail.com>
>>> > >> > > >> wrote:
>>> > >> > > >>
>>> > >> > > >> > +1 please.
>>> > >> > > >> >
>>> > >> > > >> >
>>> > >> > > >> > On Tue, Feb 18, 2014 at 6:04 PM, Michael
Ernest <
>>> > >> > > mfernest@cloudera.com
>>> > >> > > >> > >wrote:
>>> > >> > > >> >
>>> > >> > > >> > > +1
>>> > >> > > >> > >
>>> > >> > > >> > >
>>> > >> > > >> > > On Tue, Feb 18, 2014 at 8:24 AM, Heiko
Braun <
>>> > >> > > >> ike.braun@googlemail.com
>>> > >> > > >> > > >wrote:
>>> > >> > > >> > >
>>> > >> > > >> > > >
>>> > >> > > >> > > >
>>> > >> > > >> > > > Wouldn't it be better to move
the github messages to a
>>> > >> dedicated
>>> > >> > > >> email
>>> > >> > > >> > > > list?
>>> > >> > > >> > > >
>>> > >> > > >> > > > Regards, Heiko
>>> > >> > > >> > > >
>>> > >> > > >> > >
>>> > >> > > >> > >
>>> > >> > > >> > >
>>> > >> > > >> > > --
>>> > >> > > >> > > Michael Ernest
>>> > >> > > >> > > Sr. Solutions Consultant
>>> > >> > > >> > > West Coast
>>> > >> > > >> > >
>>> > >> > > >> >
>>> > >> > > >>
>>> > >> > > >
>>> > >> > >
>>> > >> >
>>> > >>
>>> >
>>> >
>>>
>

Mime
View raw message