spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject spark git commit: [SPARK-7265] Improving documentation for Spark SQL Hive support
Date Fri, 19 Jun 2015 12:07:10 GMT
Repository: spark
Updated Branches:
  refs/heads/master 93360dc3c -> ebd363aec

[SPARK-7265] Improving documentation for Spark SQL Hive support

Please review this pull request.

Author: Jihong MA <>

Closes #5933 from JihongMA/SPARK-7265 and squashes the following commits:

dfaa971 [Jihong MA] SPARK-7265 minor fix of the content
ace454d [Jihong MA] SPARK-7265 take out PySpark on YARN limitation
9ea0832 [Jihong MA] Merge remote-tracking branch 'upstream/master'
d5bf3f5 [Jihong MA] Merge remote-tracking branch 'upstream/master'
7b842e6 [Jihong MA] Merge remote-tracking branch 'upstream/master'
9c84695 [Jihong MA] SPARK-7265 address review comment
a399aa6 [Jihong MA] SPARK-7265 Improving documentation for Spark SQL Hive support


Branch: refs/heads/master
Commit: ebd363aecde977511469d47fb1ea7cb5df3c3541
Parents: 93360dc
Author: Jihong MA <>
Authored: Fri Jun 19 14:05:11 2015 +0200
Committer: Sean Owen <>
Committed: Fri Jun 19 14:06:49 2015 +0200

 docs/ | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/docs/ b/docs/
index 9b5ea39..26c036f 100644
--- a/docs/
+++ b/docs/
@@ -1445,7 +1445,12 @@ This command builds a new assembly jar that includes Hive. Note that
this Hive a
 on all of the worker nodes, as they will need access to the Hive serialization and deserialization
 (SerDes) in order to access data stored in Hive.
-Configuration of Hive is done by placing your `hive-site.xml` file in `conf/`.
+Configuration of Hive is done by placing your `hive-site.xml` file in `conf/`. Please note
when running 
+the query on a YARN cluster (`yarn-cluster` mode), the `datanucleus` jars under the `lib_managed/jars`
+and `hive-site.xml` under `conf/` directory need to be available on the driver and all executors
launched by the
+YARN cluster. The convenient way to do this is adding them through the `--jars` option and
`--file` option of the
+`spark-submit` command.
 <div class="codetabs">

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message