spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From r...@apache.org
Subject spark git commit: [SPARK-10711] [SPARKR] Do not assume spark.submit.deployMode is always set
Date Tue, 22 Sep 2015 04:10:13 GMT
Repository: spark
Updated Branches:
  refs/heads/branch-1.5 86f9a3513 -> bb8e481bc


[SPARK-10711] [SPARKR] Do not assume spark.submit.deployMode is always set

In ```RUtils.sparkRPackagePath()``` we
1. Call ``` sys.props("spark.submit.deployMode")``` which returns null if ```spark.submit.deployMode```
is not suet
2. Call ``` sparkConf.get("spark.submit.deployMode")``` which throws ```NoSuchElementException```
if ```spark.submit.deployMode``` is not set. This patch simply passes a default value ("cluster")
for ```spark.submit.deployMode```.

cc rxin

Author: Hossein <hossein@databricks.com>

Closes #8832 from falaki/SPARK-10711.

(cherry picked from commit c986e933a900602af47966bd41edb2116c421a39)
Signed-off-by: Reynold Xin <rxin@databricks.com>


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/bb8e481b
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/bb8e481b
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/bb8e481b

Branch: refs/heads/branch-1.5
Commit: bb8e481bc6bb2a2a0452a89ed43de30c772c92b3
Parents: 86f9a35
Author: Hossein <hossein@databricks.com>
Authored: Mon Sep 21 21:09:59 2015 -0700
Committer: Reynold Xin <rxin@databricks.com>
Committed: Mon Sep 21 21:10:06 2015 -0700

----------------------------------------------------------------------
 core/src/main/scala/org/apache/spark/api/r/RUtils.scala | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/bb8e481b/core/src/main/scala/org/apache/spark/api/r/RUtils.scala
----------------------------------------------------------------------
diff --git a/core/src/main/scala/org/apache/spark/api/r/RUtils.scala b/core/src/main/scala/org/apache/spark/api/r/RUtils.scala
index 93b3bea..daad5b5 100644
--- a/core/src/main/scala/org/apache/spark/api/r/RUtils.scala
+++ b/core/src/main/scala/org/apache/spark/api/r/RUtils.scala
@@ -45,7 +45,7 @@ private[spark] object RUtils {
         (sys.props("spark.master"), sys.props("spark.submit.deployMode"))
       } else {
         val sparkConf = SparkEnv.get.conf
-        (sparkConf.get("spark.master"), sparkConf.get("spark.submit.deployMode"))
+        (sparkConf.get("spark.master"), sparkConf.get("spark.submit.deployMode", "client"))
       }
 
     val isYarnCluster = master != null && master.contains("yarn") && deployMode
== "cluster"


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message