Mailing-List: contact dev-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Date: Thu, 29 Jan 2015 17:13:35 +0000 (UTC)
From: "Brock Noland (JIRA)" <jira@apache.org>
To: hive-dev@hadoop.apache.org
Message-ID: <JIRA.12763635.1419398068000.206162.1422551615375@Atlassian.JIRA>
In-Reply-To: <JIRA.12763635.1419398068000@Atlassian.JIRA>
References: <JIRA.12763635.1419398068000@Atlassian.JIRA>
 <JIRA.12763635.1419398068203@arcas>
Subject: [jira] [Commented] (HIVE-9211) Research on build mini HoS cluster
 on YARN for unit test[Spark Branch]
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HIVE-9211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14297147#comment-14297147 ] 

Brock Noland commented on HIVE-9211:
------------------------------------

Hi [~chengxiang li],

When we [moved over to a none SNAPSHOT version of Spark|https://github.com/apache/hive/commit/dab416b2c492d22ab76fa2782f434d165c1144ab] I used a tarball which does not include the hadoop jars in the spark assembly. This can been seen by extracting the spark assembly in [our tarball|http://d3jw87u4immizc.cloudfront.net/spark-tarball/spark-${spark.version}-bin-hadoop2-without-hive.tgz]. As such you'll see I put {{\$\{test.hive.hadoop.classpath\}}} in the classpath to replace the missing hadoop jars from the spark assembly.

As such, I have the following questions:

# which class are you not finding that is required?
# when you say you need the latest branch-1.2, do you mean a released version of spark? We can have a snapshot on the spark branch but not on trunk.


> Research on build mini HoS cluster on YARN for unit test[Spark Branch]
> ----------------------------------------------------------------------
>
>                 Key: HIVE-9211
>                 URL: https://issues.apache.org/jira/browse/HIVE-9211
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Spark
>            Reporter: Chengxiang Li
>            Assignee: Chengxiang Li
>              Labels: Spark-M5
>         Attachments: HIVE-9211.1-spark.patch, HIVE-9211.2-spark.patch, HIVE-9211.2-spark.patch, HIVE-9211.3-spark.patch, HIVE-9211.4-spark.patch
>
>
> HoS on YARN is a common use case in product environment, we'd better enable unit test for this case. 


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)