Mailing-List: contact dev-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Date: Thu, 31 Jul 2014 20:43:39 +0000 (UTC)
From: "Xuefu Zhang (JIRA)" <jira@apache.org>
To: hive-dev@hadoop.apache.org
Message-ID: <JIRA.12725066.1404327749900.89635.1406839419268@arcas>
In-Reply-To: <JIRA.12725066.1404327749900@arcas>
References: <JIRA.12725066.1404327749900@arcas>
Subject: [jira] [Commented] (HIVE-7334) Create SparkShuffler, shuffling data
 between map-side data processing and reduce-side processing
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HIVE-7334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081430#comment-14081430 ] 

Xuefu Zhang commented on HIVE-7334:
-----------------------------------

[~lirui] Please feel free to create smaller JIRAs to enable sorting in Hive on Spark. Here are some ideas:

1. Complete SortByShuffler
2. Add logic in SparkCompiler to generate SparkEdgeProperty with right sorting property.
3. Add logic in SparkPlanGenerator to generate plan with right shuffle type.
4. Test Hive's sorting related queries to make sure they work. File JIRAs for problems found.

Also, please take a look at the link [~rxin] pointed out above to see if we can benefit in any way.

> Create SparkShuffler, shuffling data between map-side data processing and reduce-side processing
> ------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-7334
>                 URL: https://issues.apache.org/jira/browse/HIVE-7334
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Xuefu Zhang
>            Assignee: Rui Li
>         Attachments: HIVE-7334.patch
>
>
> Please refer to the design spec.


--
This message was sent by Atlassian JIRA
(v6.2#6252)