spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From marmb...@apache.org
Subject spark git commit: [SPARK-11208][SQL] Filter out 'hive.metastore.rawstore.impl' from executionHive temporary config
Date Wed, 21 Oct 2015 21:12:27 GMT
Repository: spark
Updated Branches:
  refs/heads/master 3afe448d3 -> a37cd8704


[SPARK-11208][SQL] Filter out 'hive.metastore.rawstore.impl' from executionHive temporary
config

The executionHive assumed to be a standard meta store located in temporary directory as a
derby db. But hive.metastore.rawstore.impl was not filtered out so any custom implementation
of the metastore with other storage properties (not JDO) will persist that temporary functions.
CassandraHiveMetaStore from DataStax Enterprise is one of examples.

Author: Artem Aliev <artem.aliev@datastax.com>

Closes #9178 from artem-aliev/SPARK-11208.


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/a37cd870
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/a37cd870
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/a37cd870

Branch: refs/heads/master
Commit: a37cd870489d7ff85455bf9b57d9aa5329514058
Parents: 3afe448
Author: Artem Aliev <artem.aliev@datastax.com>
Authored: Wed Oct 21 14:12:00 2015 -0700
Committer: Michael Armbrust <michael@databricks.com>
Committed: Wed Oct 21 14:12:07 2015 -0700

----------------------------------------------------------------------
 .../src/main/scala/org/apache/spark/sql/hive/HiveContext.scala    | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/a37cd870/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveContext.scala
----------------------------------------------------------------------
diff --git a/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveContext.scala b/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveContext.scala
index 61f6116..c7460d4 100644
--- a/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveContext.scala
+++ b/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveContext.scala
@@ -725,7 +725,8 @@ private[hive] object HiveContext {
     // We have to mask all properties in hive-site.xml that relates to metastore data source
     // as we used a local metastore here.
     HiveConf.ConfVars.values().foreach { confvar =>
-      if (confvar.varname.contains("datanucleus") || confvar.varname.contains("jdo")) {
+      if (confvar.varname.contains("datanucleus") || confvar.varname.contains("jdo")
+        || confvar.varname.contains("hive.metastore.rawstore.impl")) {
         propMap.put(confvar.varname, confvar.getDefaultExpr())
       }
     }


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message