spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhan Zhang (JIRA)" <>
Subject [jira] [Commented] (SPARK-20215) ReuseExchange is boken in SparkSQL
Date Fri, 14 Apr 2017 07:30:41 GMT


Zhan Zhang commented on SPARK-20215:

Seems to be fixed in SPARK-20229

> ReuseExchange is boken in SparkSQL
> ----------------------------------
>                 Key: SPARK-20215
>                 URL:
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 2.1.0
>            Reporter: Zhan Zhang
>            Priority: Minor
> Currently if we have query like: A join B Union A join C... with the same join key. Table
A will be scanned multiple times in sql. It is because the megastoreRelation are not shared
by two joins, and ExprId is different.  canonicalized in Expression will not be able to unify
them and two Exchange will not compatible and cannot be reused.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message