spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "zhoukang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SPARK-22539) Add second order for rangepartitioner since partition number may be small if the specified key is skewed
Date Thu, 16 Nov 2017 07:31:01 GMT

     [ https://issues.apache.org/jira/browse/SPARK-22539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

zhoukang updated SPARK-22539:
-----------------------------
    Description: 
The rangepartitioner generated from shuffle exchange may cause partiton skew if sort key is
skewed.
We can add second order for rangepartitioner since partition number may be small if the specified
key is skewed.
This improvement comes from real case.

  was:
The rangepartitioner generated from shuffle exchange may cause partiton skew if sort key is
skewed.
We can add second order for rangepartitioner since partition number may be small if the specified
key is skewed.


> Add second order for rangepartitioner since partition number may be small if the specified
key is skewed
> --------------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-22539
>                 URL: https://issues.apache.org/jira/browse/SPARK-22539
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 2.1.0
>            Reporter: zhoukang
>
> The rangepartitioner generated from shuffle exchange may cause partiton skew if sort
key is skewed.
> We can add second order for rangepartitioner since partition number may be small if the
specified key is skewed.
> This improvement comes from real case.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message