spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From srowen <>
Subject [GitHub] spark pull request: Simplify localFraction in CoalescedRDDPartitio...
Date Wed, 15 Jul 2015 08:15:14 GMT
Github user srowen commented on a diff in the pull request:
    --- Diff: core/src/main/scala/org/apache/spark/rdd/CoalescedRDD.scala ---
    @@ -58,7 +58,7 @@ private[spark] case class CoalescedRDDPartition(
           val parentPreferredLocations = rdd.context.getPreferredLocs(rdd, p.index).map(
    -    if (parents.size == 0) 0.0 else (loc.toDouble / parents.size.toDouble)
    --- End diff --
    I realize the irony in continuing the thread, and this is not specifically directed at
this PR alone: 
    No, it's quite easy to merge and our test process is automated. (My assertion here though
is that this is not faster -- it calls size == 0 -- even though I prefer it for readability
    It's that thousands of people are on the mailing list, and hundreds will spend 10 seconds
reading this, and a few committers will spend minutes or more at least, and it adds up to
hundreds of dollars of people's time, if you want to think of it that way. (Remember the project
gets 20-30 new issues a day, too, and committers only have part of each day to spend on these
things.) Your time is better spent looking into open starter JIRAs, if we're going to ask
the community to invest even this small amount of time.
    Same observation goes for people asking on user@ -- you're asking for many hours of time
of attention from people across the world every time in aggregate. Is your question worth
$1000? It may literally cost that. Alternatively, think of how valuable a good answer is in
time saved!

If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at or file a JIRA ticket
with INFRA.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message