spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From marmb...@apache.org
Subject spark git commit: [SPARK-6858][SQL] Register Java HashMap for SparkSqlSerializer
Date Sat, 11 Apr 2015 21:51:02 GMT
Repository: spark
Updated Branches:
  refs/heads/master 3ceb810aa -> 198cf2a3f


[SPARK-6858][SQL] Register Java HashMap for SparkSqlSerializer

Since now kyro serializer is used for `GeneralHashedRelation` whether kyro is enabled or not,
it is better to register Java `HashMap` in `SparkSqlSerializer`.

Author: Liang-Chi Hsieh <viirya@gmail.com>

Closes #5465 from viirya/register_hashmap and squashes the following commits:

9062601 [Liang-Chi Hsieh] Register Java HashMap for SparkSqlSerializer.


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/198cf2a3
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/198cf2a3
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/198cf2a3

Branch: refs/heads/master
Commit: 198cf2a3fa9babb5e8b7b44da0471b63b9f3ec04
Parents: 3ceb810
Author: Liang-Chi Hsieh <viirya@gmail.com>
Authored: Sat Apr 11 14:50:50 2015 -0700
Committer: Michael Armbrust <michael@databricks.com>
Committed: Sat Apr 11 14:50:50 2015 -0700

----------------------------------------------------------------------
 .../scala/org/apache/spark/sql/execution/SparkSqlSerializer.scala  | 2 ++
 1 file changed, 2 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/198cf2a3/sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlSerializer.scala
----------------------------------------------------------------------
diff --git a/sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlSerializer.scala
b/sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlSerializer.scala
index 347e2f4..914f387 100644
--- a/sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlSerializer.scala
+++ b/sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlSerializer.scala
@@ -18,6 +18,7 @@
 package org.apache.spark.sql.execution
 
 import java.nio.ByteBuffer
+import java.util.{HashMap => JavaHashMap}
 
 import org.apache.spark.sql.types.Decimal
 
@@ -54,6 +55,7 @@ private[sql] class SparkSqlSerializer(conf: SparkConf) extends KryoSerializer(co
     kryo.register(classOf[org.apache.spark.util.collection.OpenHashSet[_]],
                   new OpenHashSetSerializer)
     kryo.register(classOf[Decimal])
+    kryo.register(classOf[JavaHashMap[_, _]])
 
     kryo.setReferences(false)
     kryo


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message