zeppelin-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From m...@apache.org
Subject incubator-zeppelin git commit: ZEPPELIN-631 Allow user to configure spark.yarn.dist.files
Date Tue, 23 Feb 2016 03:11:19 GMT
Repository: incubator-zeppelin
Updated Branches:
  refs/heads/master d2f9e6475 -> 8cdf71522


ZEPPELIN-631 Allow user to configure spark.yarn.dist.files

### What is this PR for?
This enables to append values into spark.yarn.dist.files

### What type of PR is it?
Bug Fix

### Todos
* [x] - Fix the logic to update values of spark.yarn.dis.files

### Is there a relevant Jira issue?
ZEPPELIN-631

### How should this be tested?
TBD

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No

Author: Jongyoul Lee <jongyoul@gmail.com>

Closes #735 from jongyoul/ZEPPELIN-631 and squashes the following commits:

52f97ff [Jongyoul Lee] ZEPPELIN-631 Allow user to configure spark.yarn.dist.files - Enabled
to append values into spark.yarn.dist.files


Project: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/commit/8cdf7152
Tree: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/tree/8cdf7152
Diff: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/diff/8cdf7152

Branch: refs/heads/master
Commit: 8cdf715220cdb3034c4992d38cac96f11f9988dd
Parents: d2f9e64
Author: Jongyoul Lee <jongyoul@gmail.com>
Authored: Fri Feb 19 12:18:25 2016 +0900
Committer: Lee moon soo <moon@apache.org>
Committed: Mon Feb 22 19:14:43 2016 -0800

----------------------------------------------------------------------
 .../main/java/org/apache/zeppelin/spark/SparkInterpreter.java | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/8cdf7152/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java
----------------------------------------------------------------------
diff --git a/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java b/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java
index be891d8..a905fb7 100644
--- a/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java
+++ b/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java
@@ -327,7 +327,12 @@ public class SparkInterpreter extends Interpreter {
     }
     pythonLibUris.trimToSize();
     if (pythonLibs.length == pythonLibUris.size()) {
-      conf.set("spark.yarn.dist.files", Joiner.on(",").join(pythonLibUris));
+      try {
+        String confValue = conf.get("spark.yarn.dist.files");
+        conf.set("spark.yarn.dist.files", confValue + "," + Joiner.on(",").join(pythonLibUris));
+      } catch (NoSuchElementException e) {
+        conf.set("spark.yarn.dist.files", Joiner.on(",").join(pythonLibUris));
+      }
       if (!useSparkSubmit()) {
         conf.set("spark.files", conf.get("spark.yarn.dist.files"));
       }


Mime
View raw message