spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yin Huai (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (SPARK-19620) Incorrect exchange coordinator Id in physical plan
Date Fri, 10 Mar 2017 19:16:04 GMT

     [ https://issues.apache.org/jira/browse/SPARK-19620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Yin Huai reassigned SPARK-19620:
--------------------------------

    Assignee: Carson Wang

> Incorrect exchange coordinator Id in physical plan
> --------------------------------------------------
>
>                 Key: SPARK-19620
>                 URL: https://issues.apache.org/jira/browse/SPARK-19620
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 2.1.0
>            Reporter: Carson Wang
>            Assignee: Carson Wang
>            Priority: Minor
>             Fix For: 2.2.0
>
>
> When adaptive execution is enabled, an exchange coordinator is used to in the Exchange
operators. For Join, the same exchange coordinator is used for its two Exchanges. But the
physical plan shows two different coordinator Ids which is confusing.
> Here is an example:
> {code}
> == Physical Plan ==
> *Project [key1#3L, value2#12L]
> +- *SortMergeJoin [key1#3L], [key2#11L], Inner
>    :- *Sort [key1#3L ASC NULLS FIRST], false, 0
>    :  +- Exchange(coordinator id: 1804587700) hashpartitioning(key1#3L, 10), coordinator[target
post-shuffle partition size: 67108864]
>    :     +- *Project [(id#0L % 500) AS key1#3L]
>    :        +- *Filter isnotnull((id#0L % 500))
>    :           +- *Range (0, 1000, step=1, splits=Some(10))
>    +- *Sort [key2#11L ASC NULLS FIRST], false, 0
>       +- Exchange(coordinator id: 793927319) hashpartitioning(key2#11L, 10), coordinator[target
post-shuffle partition size: 67108864]
>          +- *Project [(id#8L % 500) AS key2#11L, id#8L AS value2#12L]
>             +- *Filter isnotnull((id#8L % 500))
>                +- *Range (0, 1000, step=1, splits=Some(10))
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message