Mailing-List: contact dev-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Date: Wed, 26 Nov 2014 05:31:14 +0000 (UTC)
From: "Xuefu Zhang (JIRA)" <jira@apache.org>
To: hive-dev@hadoop.apache.org
Message-ID: <JIRA.12754699.1415779273000.23951.1416979874569@Atlassian.JIRA>
In-Reply-To: <JIRA.12754699.1415779273000@Atlassian.JIRA>
References: <JIRA.12754699.1415779273000@Atlassian.JIRA>
 <JIRA.12754699.1415779273054@arcas>
Subject: [jira] [Commented] (HIVE-8836) Enable automatic tests with remote
 spark client [Spark Branch]
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HIVE-8836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225779#comment-14225779 ] 

Xuefu Zhang commented on HIVE-8836:
-----------------------------------

Many test failures above are shown as plan diff, somehting like: 
{code}
211c211
<         Reducer 2 <- Map 1 (GROUP, 3)
---
>         Reducer 2 <- Map 1 (GROUP, 1)
{code}
This can be explained by the fact that the current algorithm determining the number of reducers has included the factors such as number of executors in the cluster. Because we switched to local-cluster with two nodes, the number of reducers is different from previous value. This is not good, because every time we change the testing cluster, we might have a different plan.

> Enable automatic tests with remote spark client [Spark Branch]
> --------------------------------------------------------------
>
>                 Key: HIVE-8836
>                 URL: https://issues.apache.org/jira/browse/HIVE-8836
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Spark
>    Affects Versions: spark-branch
>            Reporter: Chengxiang Li
>            Assignee: Rui Li
>              Labels: Spark-M3
>             Fix For: spark-branch
>
>         Attachments: HIVE-8836.1-spark.patch, HIVE-8836.2-spark.patch, HIVE-8836.3-spark.patch, HIVE-8836.4-spark.patch, HIVE-8836.5-spark.patch, HIVE-8836.6-spark.patch, HIVE-8836.6-spark.patch
>
>
> In real production environment, remote spark client should be used to submit spark job for Hive mostly, we should enable automatic test with remote spark client to make sure the Hive feature workable with it.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)