hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcelo Vanzin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-15302) Relax the requirement that HoS needs Spark built w/o Hive
Date Wed, 30 Nov 2016 03:33:59 GMT

    [ https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15707398#comment-15707398
] 

Marcelo Vanzin commented on HIVE-15302:
---------------------------------------

You don't need to use the archive. You can use {{spark.yarn.jars}}, for example; that doesn't
require an archive. I don't remember the exact requirements for the archive; it's documented
in Spark's documentation.

The recommended is either HDFS, or having it on every node and using a "local:" URI to tell
Spark to not upload anything.

I'm hoping that you'll be using maven to package the needed Spark dependencies instead of
the current "download tarball from somewhere" approach.

> Relax the requirement that HoS needs Spark built w/o Hive
> ---------------------------------------------------------
>
>                 Key: HIVE-15302
>                 URL: https://issues.apache.org/jira/browse/HIVE-15302
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Rui Li
>            Assignee: Rui Li
>
> This requirement becomes more and more unacceptable as SparkSQL becomes widely adopted.
Let's use this JIRA to find out how we can relax the limitation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message