hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xuefu Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-7334) Create SparkShuffler, shuffling data between map-side data processing and reduce-side processing
Date Thu, 31 Jul 2014 20:43:39 GMT

    [ https://issues.apache.org/jira/browse/HIVE-7334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081430#comment-14081430
] 

Xuefu Zhang commented on HIVE-7334:
-----------------------------------

[~lirui] Please feel free to create smaller JIRAs to enable sorting in Hive on Spark. Here
are some ideas:

1. Complete SortByShuffler
2. Add logic in SparkCompiler to generate SparkEdgeProperty with right sorting property.
3. Add logic in SparkPlanGenerator to generate plan with right shuffle type.
4. Test Hive's sorting related queries to make sure they work. File JIRAs for problems found.

Also, please take a look at the link [~rxin] pointed out above to see if we can benefit in
any way.

> Create SparkShuffler, shuffling data between map-side data processing and reduce-side
processing
> ------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-7334
>                 URL: https://issues.apache.org/jira/browse/HIVE-7334
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Xuefu Zhang
>            Assignee: Rui Li
>         Attachments: HIVE-7334.patch
>
>
> Please refer to the design spec.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message