zeppelin-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From m...@apache.org
Subject incubator-zeppelin git commit: Set any parameter to SparkConf() from Interpreter GUI
Date Mon, 30 Mar 2015 13:29:00 GMT
Repository: incubator-zeppelin
Updated Branches:
  refs/heads/master 2498e5df1 -> 77c59e4f0


Set any parameter to SparkConf() from Interpreter GUI

Only "spark.*" we propagated from Interpreter GUI but there are external tools that require
custom configuration, i.e [couchbase-spark-connector](https://github.com/couchbaselabs/couchbase-spark-connector/wiki/Connection-Management#connecting-to-one-bucket)

This PR removes restriction.

Author: Alexander Bezzubov <abezzubov@nflabs.com>

Closes #5 from bzz/set-any-spark-conf-parapeters and squashes the following commits:

6c335e2 [Alexander Bezzubov] Spark Interpreter now sets any paramenter to SparkConf


Project: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/commit/77c59e4f
Tree: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/tree/77c59e4f
Diff: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/diff/77c59e4f

Branch: refs/heads/master
Commit: 77c59e4f07cd6807bd8efe4bfa6a90fda6fea88d
Parents: 2498e5d
Author: Alexander Bezzubov <abezzubov@nflabs.com>
Authored: Thu Mar 26 18:10:07 2015 +0900
Committer: Lee moon soo <moon@apache.org>
Committed: Mon Mar 30 22:28:55 2015 +0900

----------------------------------------------------------------------
 .../com/nflabs/zeppelin/spark/SparkInterpreter.java | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/77c59e4f/spark/src/main/java/com/nflabs/zeppelin/spark/SparkInterpreter.java
----------------------------------------------------------------------
diff --git a/spark/src/main/java/com/nflabs/zeppelin/spark/SparkInterpreter.java b/spark/src/main/java/com/nflabs/zeppelin/spark/SparkInterpreter.java
index 2701e30..6177322 100644
--- a/spark/src/main/java/com/nflabs/zeppelin/spark/SparkInterpreter.java
+++ b/spark/src/main/java/com/nflabs/zeppelin/spark/SparkInterpreter.java
@@ -241,14 +241,10 @@ public class SparkInterpreter extends Interpreter {
 
     for (Object k : intpProperty.keySet()) {
       String key = (String) k;
-      if (key.startsWith("spark.")) {
-        Object value = intpProperty.get(key);
-        if (value != null
-            && value instanceof String
-            && !((String) value).trim().isEmpty()) {
-          logger.debug(String.format("SparkConf: key = [%s], value = [%s]", key, value));
-          conf.set(key, (String) value);
-        }
+      Object value = intpProperty.get(key);
+      if (!isEmptyString(value)) {
+        logger.debug(String.format("SparkConf: key = [%s], value = [%s]", key, value));
+        conf.set(key, (String) value);
       }
     }
 
@@ -256,6 +252,10 @@ public class SparkInterpreter extends Interpreter {
     return sparkContext;
   }
 
+  public static boolean isEmptyString(Object val) {
+    return val instanceof String && ((String) val).trim().isEmpty();
+  }
+
   public static String getSystemDefault(
       String envName,
       String propertyName,


Mime
View raw message