spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From pwend...@apache.org
Subject [1/4] git commit: Allow configuration to be printed in logs for diagnosis.
Date Tue, 07 Jan 2014 08:54:35 GMT
Updated Branches:
  refs/heads/master a862cafac -> c3cf0475e


Allow configuration to be printed in logs for diagnosis.


Project: http://git-wip-us.apache.org/repos/asf/incubator-spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-spark/commit/b84dc780
Tree: http://git-wip-us.apache.org/repos/asf/incubator-spark/tree/b84dc780
Diff: http://git-wip-us.apache.org/repos/asf/incubator-spark/diff/b84dc780

Branch: refs/heads/master
Commit: b84dc780d3ad3fc2f0f1e9ba3fd1676447c65e96
Parents: b301881
Author: Prashant Sharma <prashant.s@imaginea.com>
Authored: Mon Jan 6 15:51:32 2014 +0530
Committer: Prashant Sharma <prashant.s@imaginea.com>
Committed: Tue Jan 7 13:01:43 2014 +0530

----------------------------------------------------------------------
 core/src/main/scala/org/apache/spark/SparkContext.scala | 4 ++++
 docs/configuration.md                                   | 7 +++++++
 2 files changed, 11 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-spark/blob/b84dc780/core/src/main/scala/org/apache/spark/SparkContext.scala
----------------------------------------------------------------------
diff --git a/core/src/main/scala/org/apache/spark/SparkContext.scala b/core/src/main/scala/org/apache/spark/SparkContext.scala
index e80e43a..99dcced 100644
--- a/core/src/main/scala/org/apache/spark/SparkContext.scala
+++ b/core/src/main/scala/org/apache/spark/SparkContext.scala
@@ -116,6 +116,10 @@ class SparkContext(
     throw new SparkException("An application must be set in your configuration")
   }
 
+  if (conf.get("spark.log-conf", "false").toBoolean) {
+    logInfo("Spark configuration:\n" + conf.toDebugString)
+  }
+
   // Set Spark driver host and port system properties
   conf.setIfMissing("spark.driver.host", Utils.localHostName())
   conf.setIfMissing("spark.driver.port", "0")

http://git-wip-us.apache.org/repos/asf/incubator-spark/blob/b84dc780/docs/configuration.md
----------------------------------------------------------------------
diff --git a/docs/configuration.md b/docs/configuration.md
index 8a8857b..f00fe05 100644
--- a/docs/configuration.md
+++ b/docs/configuration.md
@@ -403,6 +403,13 @@ Apart from these, the following properties are also available, and may
be useful
     How many times slower a task is than the median to be considered for speculation.
   </td>
 </tr>
+<tr>
+  <td>spark.log-conf</td>
+  <td>false</td>
+  <td>
+    Log the supplied SparkConf as INFO at start of spark context.
+  </td>
+</tr>
 </table>
 
 ## Viewing Spark Properties


Mime
View raw message