spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From srowen <>
Subject [GitHub] spark issue #19764: [SPARK-22539][SQL] Add second order for rangepartitioner...
Date Tue, 02 Jan 2018 15:13:41 GMT
Github user srowen commented on the issue:
    I'm probably not qualified to review this. I don't think you addressed Herman's question.
It wasn't about ordering or whether the same exact row maps to the same partition, but whether
all values for a key map to the same partition. I believe that's part of the contract here.
If it doesn't do that, then, I don't see how it solves the problem you're trying to solve.
Skew is inherently a problem if you promise to put all values for a key together.


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message