spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From marmb...@apache.org
Subject spark git commit: [SPARK-8658][SQL][FOLLOW-UP] AttributeReference's equals method compares all the members
Date Tue, 17 Nov 2015 19:24:15 GMT
Repository: spark
Updated Branches:
  refs/heads/branch-1.6 167ea61a6 -> 933f025ce


[SPARK-8658][SQL][FOLLOW-UP] AttributeReference's equals method compares all the members

Based on the comment of cloud-fan in https://github.com/apache/spark/pull/9216, update the
AttributeReference's hashCode function by including the hashCode of the other attributes including
name, nullable and qualifiers.

Here, I am not 100% sure if we should include name in the hashCode calculation, since the
original hashCode calculation does not include it.

marmbrus cloud-fan Please review if the changes are good.

Author: gatorsmile <gatorsmile@gmail.com>

Closes #9761 from gatorsmile/hashCodeNamedExpression.

(cherry picked from commit 0158ff7737d10e68be2e289533241da96b496e89)
Signed-off-by: Michael Armbrust <michael@databricks.com>


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/933f025c
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/933f025c
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/933f025c

Branch: refs/heads/branch-1.6
Commit: 933f025ce7cc58a06f4e0c331e8e85da02909090
Parents: 167ea61
Author: gatorsmile <gatorsmile@gmail.com>
Authored: Tue Nov 17 11:23:54 2015 -0800
Committer: Michael Armbrust <michael@databricks.com>
Committed: Tue Nov 17 11:24:08 2015 -0800

----------------------------------------------------------------------
 .../spark/sql/catalyst/expressions/namedExpressions.scala      | 5 ++++-
 .../catalyst/expressions/SubexpressionEliminationSuite.scala   | 6 +++++-
 2 files changed, 9 insertions(+), 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/933f025c/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/namedExpressions.scala
----------------------------------------------------------------------
diff --git a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/namedExpressions.scala
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/namedExpressions.scala
index e3dadda..00b7970 100644
--- a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/namedExpressions.scala
+++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/namedExpressions.scala
@@ -212,9 +212,12 @@ case class AttributeReference(
   override def hashCode: Int = {
     // See http://stackoverflow.com/questions/113511/hash-code-implementation
     var h = 17
-    h = h * 37 + exprId.hashCode()
+    h = h * 37 + name.hashCode()
     h = h * 37 + dataType.hashCode()
+    h = h * 37 + nullable.hashCode()
     h = h * 37 + metadata.hashCode()
+    h = h * 37 + exprId.hashCode()
+    h = h * 37 + qualifiers.hashCode()
     h
   }
 

http://git-wip-us.apache.org/repos/asf/spark/blob/933f025c/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/SubexpressionEliminationSuite.scala
----------------------------------------------------------------------
diff --git a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/SubexpressionEliminationSuite.scala
b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/SubexpressionEliminationSuite.scala
index 9de066e..a61297b 100644
--- a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/SubexpressionEliminationSuite.scala
+++ b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/SubexpressionEliminationSuite.scala
@@ -25,14 +25,18 @@ class SubexpressionEliminationSuite extends SparkFunSuite {
     val a: AttributeReference = AttributeReference("name", IntegerType)()
     val b1 = a.withName("name2").withExprId(id)
     val b2 = a.withExprId(id)
+    val b3 = a.withQualifiers("qualifierName" :: Nil)
 
     assert(b1 != b2)
     assert(a != b1)
     assert(b1.semanticEquals(b2))
     assert(!b1.semanticEquals(a))
     assert(a.hashCode != b1.hashCode)
-    assert(b1.hashCode == b2.hashCode)
+    assert(b1.hashCode != b2.hashCode)
     assert(b1.semanticHash() == b2.semanticHash())
+    assert(a != b3)
+    assert(a.hashCode != b3.hashCode)
+    assert(a.semanticEquals(b3))
   }
 
   test("Expression Equivalence - basic") {


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message