spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From cloud-fan <...@git.apache.org>
Subject [GitHub] spark pull request #20541: [SPARK-23356][SQL]Pushes Project to both sides of...
Date Thu, 08 Feb 2018 09:18:07 GMT
Github user cloud-fan commented on a diff in the pull request:

    https://github.com/apache/spark/pull/20541#discussion_r166870474
  
    --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
    @@ -400,13 +400,24 @@ object PushProjectionThroughUnion extends Rule[LogicalPlan] with
PredicateHelper
         // Push down deterministic projection through UNION ALL
         case p @ Project(projectList, Union(children)) =>
           assert(children.nonEmpty)
    -      if (projectList.forall(_.deterministic)) {
    -        val newFirstChild = Project(projectList, children.head)
    +      val (deterministicList, nonDeterministic) = projectList.partition(_.deterministic)
    +
    +      if (deterministicList.nonEmpty) {
    +        val newFirstChild = Project(deterministicList, children.head)
             val newOtherChildren = children.tail.map { child =>
               val rewrites = buildRewrites(children.head, child)
    -          Project(projectList.map(pushToRight(_, rewrites)), child)
    +          Project(deterministicList.map(pushToRight(_, rewrites)), child)
    --- End diff --
    
    do we push `a + 1` to union children? or just `a`?


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message