spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Wendell <pwend...@gmail.com>
Subject Re: SPARK-942 patch review
Date Wed, 26 Feb 2014 02:32:30 GMT
Hey Andrew,

Indeed, sometimes there are patches that sit around a while and in
this case it can be because it's unclear to the reviewers whether they
are features worth having - or just by accident.

To put things in perspective, Spark merges about 80% of the proposed
patches (if you look we are on around 600 since moving to the new repo
with 100 not merged) - so in general we try hard to be very supportive
of community patches, much more than other projects in this space.

- Patrick

On Mon, Feb 24, 2014 at 1:39 PM, Matei Zaharia <matei.zaharia@gmail.com> wrote:
> Thanks for bringing this up. One issue that makes this harder is that old inactive PRs
on GitHub are not really getting closed, so active ones might be lost between those. For now
please just post on the dev list if your PR is being ignored. We'll implement some kind of
cleanup (at least manually) to close the old ones.
>
> Matei
>
> On Feb 24, 2014, at 1:30 PM, Andrew Ash <andrew@andrewash.com> wrote:
>
>> Yep that's the one thanks! That's quite a few more people than I thought
>>
>> Sent from my mobile phone
>> On Feb 24, 2014 1:20 PM, "Nan Zhu" <zhunanmcgill@gmail.com> wrote:
>>
>>> Do you mean this
>>> https://cwiki.apache.org/confluence/display/SPARK/Committers?
>>>
>>> --
>>> Nan Zhu
>>>
>>>
>>> On Monday, February 24, 2014 at 4:18 PM, Andrew Ash wrote:
>>>
>>>> Would love to have a discussion since I know the core contributors are
>>>> facing a barrage of PRs and things are falling through the cracks.
>>>>
>>>> Is there a list of who can commit to core Spark somewhere? Maybe that
>>> list
>>>> should be expanded or there should be a rotation of PR duty of some sort.
>>>>
>>>> One of the perils of having a vibrant, organic community is that you get
>>>> way more contributions than you expected!
>>>>
>>>>
>>>> On Mon, Feb 24, 2014 at 1:16 PM, Nan Zhu <zhunanmcgill@gmail.com(mailto:
>>> zhunanmcgill@gmail.com)> wrote:
>>>>
>>>>> yet another email about forgotten PR
>>>>>
>>>>> I think Sean would like to start some discussion on the current
>>> situation
>>>>> where committers are facing a flood of PRs recently (as he said in the
>>>>> discussion thread about how to prevent the blob of RDD API)?
>>>>>
>>>>> Best,
>>>>>
>>>>> --
>>>>> Nan Zhu
>>>>>
>>>>>
>>>>> On Monday, February 24, 2014 at 4:07 PM, Andrew Ash wrote:
>>>>>
>>>>>> Hi Spark devs,
>>>>>>
>>>>>> Kyle identified a deficiency in Spark where generating iterators
are
>>>>>> unrolled into memory and then flushed to disk rather than sent
>>> straight
>>>>>>
>>>>>
>>>>> to
>>>>>> disk when possible.
>>>>>>
>>>>>> He's had a patch sitting ready for code review for quite some time
>>> now
>>>>> (100
>>>>>> days) but no response.
>>>>>>
>>>>>> Is this something that an admin would be able to review? I for one
>>> would
>>>>>> find this quite valuable.
>>>>>>
>>>>>> Thanks!
>>>>>> Andrew
>>>>>>
>>>>>>
>>>>>> https://spark-project.atlassian.net/browse/SPARK-942
>>>>>> https://github.com/apache/incubator-spark/pull/180
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>

Mime
View raw message