spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From marmb...@apache.org
Subject spark git commit: [SPARK-4828] [SQL] sum and avg on empty table should always return null
Date Fri, 12 Dec 2014 06:49:32 GMT
Repository: spark
Updated Branches:
  refs/heads/master d8cf67858 -> acb3be6bc


[SPARK-4828] [SQL] sum and avg on empty table should always return null

So the optimizations are not valid. Also I think the optimization here is rarely encounter,
so removing them will not have influence on performance.

Can we merge #3445 before I add a comparison test case from this?

Author: Daoyuan Wang <daoyuan.wang@intel.com>

Closes #3675 from adrian-wang/sumempty and squashes the following commits:

42df763 [Daoyuan Wang] sum and avg on empty table should always return null


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/acb3be6b
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/acb3be6b
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/acb3be6b

Branch: refs/heads/master
Commit: acb3be6bc5e0e793f769ec371dffeb474dc0cf21
Parents: d8cf678
Author: Daoyuan Wang <daoyuan.wang@intel.com>
Authored: Thu Dec 11 22:49:27 2014 -0800
Committer: Michael Armbrust <michael@databricks.com>
Committed: Thu Dec 11 22:49:27 2014 -0800

----------------------------------------------------------------------
 .../scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala  | 2 --
 1 file changed, 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/acb3be6b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
----------------------------------------------------------------------
diff --git a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
index f164a6c..c492348 100644
--- a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
+++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
@@ -203,8 +203,6 @@ object NullPropagation extends Rule[LogicalPlan] {
   def apply(plan: LogicalPlan): LogicalPlan = plan transform {
     case q: LogicalPlan => q transformExpressionsUp {
       case e @ Count(Literal(null, _)) => Cast(Literal(0L), e.dataType)
-      case e @ Sum(Literal(c, _)) if c == 0 => Cast(Literal(0L), e.dataType)
-      case e @ Average(Literal(c, _)) if c == 0 => Literal(0.0, e.dataType)
       case e @ IsNull(c) if !c.nullable => Literal(false, BooleanType)
       case e @ IsNotNull(c) if !c.nullable => Literal(true, BooleanType)
       case e @ GetItem(Literal(null, _), _) => Literal(null, e.dataType)


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message