spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-18874) First phase: Deferring the correlated predicate pull up to Optimizer phase
Date Tue, 24 Jul 2018 15:54:00 GMT

    [ https://issues.apache.org/jira/browse/SPARK-18874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16554431#comment-16554431
] 

Apache Spark commented on SPARK-18874:
--------------------------------------

User 'wangyum' has created a pull request for this issue:
https://github.com/apache/spark/pull/21863

> First phase: Deferring the correlated predicate pull up to Optimizer phase
> --------------------------------------------------------------------------
>
>                 Key: SPARK-18874
>                 URL: https://issues.apache.org/jira/browse/SPARK-18874
>             Project: Spark
>          Issue Type: Sub-task
>          Components: SQL
>            Reporter: Nattavut Sutyanyong
>            Assignee: Dilip Biswal
>            Priority: Major
>             Fix For: 2.2.0
>
>         Attachments: SPARK-18874-3.pdf
>
>
> This JIRA implements the first phase of SPARK-18455 by deferring the correlated predicate
pull up from Analyzer to Optimizer. The goal is to preserve the current functionality of subquery
in Spark 2.0 (if it works, it continues to work after this JIRA, if it does not, it won't).
The performance of subquery processing is expected to be at par with Spark 2.0.
> The representation of the LogicalPlan after Analyzer will be different after this JIRA
that it will preserve the original positions of correlated predicates in a subquery. This
new representation is a preparation work for the second phase of extending the support of
correlated subquery to cases Spark 2.0 does not support such as deep correlation, outer references
in SELECT clause.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message