hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcelo Vanzin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-15302) Relax the requirement that HoS needs Spark built w/o Hive
Date Fri, 14 Apr 2017 16:52:41 GMT

    [ https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15969242#comment-15969242
] 

Marcelo Vanzin commented on HIVE-15302:
---------------------------------------

I'm not sure which approach you're trying here, but I'd really discourage you from trying
to manually figure out the list of needed jars like that... that's what maven is for.

I've done something like this for Livy in the past by having a fake spark-submit script that
does everything that Livy needs:
https://github.com/cloudera/livy/commit/3c314b11777459e10984ab408aaf2cbd47edf6db

The test code in Livy provides the needed classpath ({{System.getProperty("java.class.path")}}),
and it all works out. You could even expand on that idea to do this outside of tests too,
by adding features to the fake spark-submit script.

> Relax the requirement that HoS needs Spark built w/o Hive
> ---------------------------------------------------------
>
>                 Key: HIVE-15302
>                 URL: https://issues.apache.org/jira/browse/HIVE-15302
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Rui Li
>            Assignee: Rui Li
>
> This requirement becomes more and more unacceptable as SparkSQL becomes widely adopted.
Let's use this JIRA to find out how we can relax the limitation.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message