spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Rosen (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SPARK-8319) Enable Tungsten shuffle manager for some shuffles that specify key orderings
Date Fri, 12 Jun 2015 00:47:01 GMT

     [ https://issues.apache.org/jira/browse/SPARK-8319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Josh Rosen updated SPARK-8319:
------------------------------
    Component/s: SQL

> Enable Tungsten shuffle manager for some shuffles that specify key orderings
> ----------------------------------------------------------------------------
>
>                 Key: SPARK-8319
>                 URL: https://issues.apache.org/jira/browse/SPARK-8319
>             Project: Spark
>          Issue Type: Improvement
>          Components: Shuffle, SQL
>            Reporter: Josh Rosen
>            Assignee: Josh Rosen
>
> The Tungsten ShuffleManager falls back to regular SortShuffleManager whenever the shuffle
dependency specifies a key ordering, but technically we only need to fall back when an aggregator
is also specified.  We should update the fallback logic to handle this case so that the Tungsten
optimizations can apply to more workloads.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message