spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sro...@apache.org
Subject spark git commit: [SPARK-7265] Improving documentation for Spark SQL Hive support
Date Fri, 19 Jun 2015 12:07:10 GMT
Repository: spark
Updated Branches:
  refs/heads/master 93360dc3c -> ebd363aec


[SPARK-7265] Improving documentation for Spark SQL Hive support

Please review this pull request.

Author: Jihong MA <linlin200605@gmail.com>

Closes #5933 from JihongMA/SPARK-7265 and squashes the following commits:

dfaa971 [Jihong MA] SPARK-7265 minor fix of the content
ace454d [Jihong MA] SPARK-7265 take out PySpark on YARN limitation
9ea0832 [Jihong MA] Merge remote-tracking branch 'upstream/master'
d5bf3f5 [Jihong MA] Merge remote-tracking branch 'upstream/master'
7b842e6 [Jihong MA] Merge remote-tracking branch 'upstream/master'
9c84695 [Jihong MA] SPARK-7265 address review comment
a399aa6 [Jihong MA] SPARK-7265 Improving documentation for Spark SQL Hive support


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/ebd363ae
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/ebd363ae
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/ebd363ae

Branch: refs/heads/master
Commit: ebd363aecde977511469d47fb1ea7cb5df3c3541
Parents: 93360dc
Author: Jihong MA <linlin200605@gmail.com>
Authored: Fri Jun 19 14:05:11 2015 +0200
Committer: Sean Owen <sowen@cloudera.com>
Committed: Fri Jun 19 14:06:49 2015 +0200

----------------------------------------------------------------------
 docs/sql-programming-guide.md | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/ebd363ae/docs/sql-programming-guide.md
----------------------------------------------------------------------
diff --git a/docs/sql-programming-guide.md b/docs/sql-programming-guide.md
index 9b5ea39..26c036f 100644
--- a/docs/sql-programming-guide.md
+++ b/docs/sql-programming-guide.md
@@ -1445,7 +1445,12 @@ This command builds a new assembly jar that includes Hive. Note that
this Hive a
 on all of the worker nodes, as they will need access to the Hive serialization and deserialization
libraries
 (SerDes) in order to access data stored in Hive.
 
-Configuration of Hive is done by placing your `hive-site.xml` file in `conf/`.
+Configuration of Hive is done by placing your `hive-site.xml` file in `conf/`. Please note
when running 
+the query on a YARN cluster (`yarn-cluster` mode), the `datanucleus` jars under the `lib_managed/jars`
directory 
+and `hive-site.xml` under `conf/` directory need to be available on the driver and all executors
launched by the
+YARN cluster. The convenient way to do this is adding them through the `--jars` option and
`--file` option of the
+`spark-submit` command.
+
 
 <div class="codetabs">
 


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message