hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brock Noland (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-9211) Research on build mini HoS cluster on YARN for unit test[Spark Branch]
Date Thu, 29 Jan 2015 17:13:35 GMT

    [ https://issues.apache.org/jira/browse/HIVE-9211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14297147#comment-14297147
] 

Brock Noland commented on HIVE-9211:
------------------------------------

Hi [~chengxiang li],

When we [moved over to a none SNAPSHOT version of Spark|https://github.com/apache/hive/commit/dab416b2c492d22ab76fa2782f434d165c1144ab]
I used a tarball which does not include the hadoop jars in the spark assembly. This can been
seen by extracting the spark assembly in [our tarball|http://d3jw87u4immizc.cloudfront.net/spark-tarball/spark-${spark.version}-bin-hadoop2-without-hive.tgz].
As such you'll see I put {{\$\{test.hive.hadoop.classpath\}}} in the classpath to replace
the missing hadoop jars from the spark assembly.

As such, I have the following questions:

# which class are you not finding that is required?
# when you say you need the latest branch-1.2, do you mean a released version of spark? We
can have a snapshot on the spark branch but not on trunk.



> Research on build mini HoS cluster on YARN for unit test[Spark Branch]
> ----------------------------------------------------------------------
>
>                 Key: HIVE-9211
>                 URL: https://issues.apache.org/jira/browse/HIVE-9211
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Spark
>            Reporter: Chengxiang Li
>            Assignee: Chengxiang Li
>              Labels: Spark-M5
>         Attachments: HIVE-9211.1-spark.patch, HIVE-9211.2-spark.patch, HIVE-9211.2-spark.patch,
HIVE-9211.3-spark.patch, HIVE-9211.4-spark.patch
>
>
> HoS on YARN is a common use case in product environment, we'd better enable unit test
for this case. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message