spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From srowen <...@git.apache.org>
Subject [GitHub] spark issue #19764: [SPARK-22539][SQL] Add second order for rangepartitioner...
Date Tue, 02 Jan 2018 15:13:41 GMT
Github user srowen commented on the issue:

    https://github.com/apache/spark/pull/19764
  
    I'm probably not qualified to review this. I don't think you addressed Herman's question.
It wasn't about ordering or whether the same exact row maps to the same partition, but whether
all values for a key map to the same partition. I believe that's part of the contract here.
If it doesn't do that, then, I don't see how it solves the problem you're trying to solve.
Skew is inherently a problem if you promise to put all values for a key together.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message