spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From van...@apache.org
Subject spark git commit: [SPARK-15760][DOCS] Add documentation for package-related configs.
Date Tue, 07 Jun 2016 16:29:27 GMT
Repository: spark
Updated Branches:
  refs/heads/branch-2.0 ead3bbdae -> f1439b90c


[SPARK-15760][DOCS] Add documentation for package-related configs.

While there, also document spark.files and spark.jars. Text is the
same as the spark-submit help text with some minor adjustments.

Author: Marcelo Vanzin <vanzin@cloudera.com>

Closes #13502 from vanzin/SPARK-15760.

(cherry picked from commit 200f01c8fb15680b5630fbd122d44f9b1d096e02)
Signed-off-by: Marcelo Vanzin <vanzin@cloudera.com>


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/f1439b90
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/f1439b90
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/f1439b90

Branch: refs/heads/branch-2.0
Commit: f1439b90c64f9c41d3dde3950602282f54d6e96d
Parents: ead3bbd
Author: Marcelo Vanzin <vanzin@cloudera.com>
Authored: Tue Jun 7 09:28:39 2016 -0700
Committer: Marcelo Vanzin <vanzin@cloudera.com>
Committed: Tue Jun 7 09:29:03 2016 -0700

----------------------------------------------------------------------
 docs/configuration.md | 47 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 47 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/f1439b90/docs/configuration.md
----------------------------------------------------------------------
diff --git a/docs/configuration.md b/docs/configuration.md
index d6471a8..32c3a92 100644
--- a/docs/configuration.md
+++ b/docs/configuration.md
@@ -380,6 +380,53 @@ Apart from these, the following properties are also available, and may
be useful
     from JVM to Python worker for every task.
   </td>
 </tr>
+<tr>
+  <td><code>spark.files</code></td>
+  <td></td>
+  <td>
+    Comma-separated list of files to be placed in the working directory of each executor.
+  </td>
+</tr>
+<tr>
+  <td><code>spark.submit.pyFiles</code></td>
+  <td></td>
+  <td>
+    Comma-separated list of .zip, .egg, or .py files to place on the PYTHONPATH for Python
apps.
+  </td>
+</tr>
+<tr>
+  <td><code>spark.jars</code></td>
+  <td></td>
+  <td>
+    Comma-separated list of local jars to include on the driver and executor classpaths.
+  </td>
+</tr>
+<tr>
+  <td><code>spark.jars.packages</code></td>
+  <td></td>
+  <td>
+    Comma-separated list of maven coordinates of jars to include on the driver and executor
+    classpaths. Will search the local maven repo, then maven central and any additional remote
+    repositories given by <code>spark.jars.ivy</code>. The format for the coordinates
should be
+    groupId:artifactId:version.
+  </td>
+</tr>
+<tr>
+  <td><code>spark.jars.excludes</code></td>
+  <td></td>
+  <td>
+    Comma-separated list of groupId:artifactId, to exclude while resolving the dependencies
+    provided in <code>spark.jars.packages</code> to avoid dependency conflicts.
+  </td>
+</tr>
+<tr>
+  <td><code>spark.jars.ivy</code></td>
+  <td></td>
+  <td>
+    Comma-separated list of additional remote repositories to search for the coordinates
given
+    with <code>spark.jars.packages</code>.
+  </td>
+</tr>
 </table>
 
 #### Shuffle Behavior


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message