spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From gengliangwang <...@git.apache.org>
Subject [GitHub] spark pull request #19362: [SPARK-22141][SQL] Propagate empty relation befor...
Date Wed, 27 Sep 2017 08:05:32 GMT
GitHub user gengliangwang opened a pull request:

    https://github.com/apache/spark/pull/19362

    [SPARK-22141][SQL] Propagate empty relation before checking Cartesian products

    ## What changes were proposed in this pull request?
    
    When inferring constraints from children, Join's condition can be simplified as None.
    For example,
    ```
    val testRelation = LocalRelation('a.int)
    val x = testRelation.as("x")
    val y = testRelation.where($"a" === 2 && !($"a" === 2)).as("y")
    x.join.where($"x.a" === $"y.a")
    ```
    The plan will become
    ```
    Join Inner
    :- LocalRelation <empty>, [a#23]
    +- LocalRelation <empty>, [a#224]
    ```
    And the Cartesian products check will throw exception for above plan.
    
    Propagate empty relation before checking Cartesian products, and the issue is resolved.
    
    ## How was this patch tested?
    
    Unit test


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/gengliangwang/spark MoveCheckCartesianProducts

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/19362.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #19362
    
----
commit 0f25a074821481e89a256824b40acf78a06841e5
Author: Wang Gengliang <ltnwgl@gmail.com>
Date:   2017-09-27T07:51:08Z

    propagate empty relation before checking Cartesian products

----


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message