spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ma...@apache.org
Subject git commit: [SPARK-2116] Load spark-defaults.conf from SPARK_CONF_DIR if set
Date Sat, 02 Aug 2014 02:02:19 GMT
Repository: spark
Updated Branches:
  refs/heads/master 3822f33f3 -> 0da07da53


[SPARK-2116] Load spark-defaults.conf from SPARK_CONF_DIR if set

If SPARK_CONF_DIR environment variable is set, search it for spark-defaults.conf.

Author: Albert Chu <chu11@llnl.gov>

Closes #1059 from chu11/SPARK-2116 and squashes the following commits:

9f3ac94 [Albert Chu] SPARK-2116: If SPARK_CONF_DIR environment variable is set, search it
for spark-defaults.conf.


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/0da07da5
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/0da07da5
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/0da07da5

Branch: refs/heads/master
Commit: 0da07da53e5466ec44c8050020cbc4b9957cb949
Parents: 3822f33
Author: Albert Chu <chu11@llnl.gov>
Authored: Fri Aug 1 19:00:38 2014 -0700
Committer: Matei Zaharia <matei@databricks.com>
Committed: Fri Aug 1 19:00:46 2014 -0700

----------------------------------------------------------------------
 .../org/apache/spark/deploy/SparkSubmitArguments.scala   | 11 +++++++++++
 1 file changed, 11 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/0da07da5/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala
----------------------------------------------------------------------
diff --git a/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala b/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala
index dd044e6..9391f24 100644
--- a/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala
+++ b/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala
@@ -86,6 +86,17 @@ private[spark] class SparkSubmitArguments(args: Seq[String]) {
   private def mergeSparkProperties(): Unit = {
     // Use common defaults file, if not specified by user
     if (propertiesFile == null) {
+      sys.env.get("SPARK_CONF_DIR").foreach { sparkConfDir =>
+        val sep = File.separator
+        val defaultPath = s"${sparkConfDir}${sep}spark-defaults.conf"
+        val file = new File(defaultPath)
+        if (file.exists()) {
+          propertiesFile = file.getAbsolutePath
+        }
+      }
+    }
+
+    if (propertiesFile == null) {
       sys.env.get("SPARK_HOME").foreach { sparkHome =>
         val sep = File.separator
         val defaultPath = s"${sparkHome}${sep}conf${sep}spark-defaults.conf"


Mime
View raw message