hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phabricator (JIRA)" <>
Subject [jira] [Commented] (HIVE-3972) Support using multiple reducer for fetching order by results
Date Tue, 05 Feb 2013 08:56:16 GMT


Phabricator commented on HIVE-3972:

njain has commented on the revision "HIVE-3972 [jira] Support using multiple reducer for fetching
order by results".

  conf/hive-default.xml.template:1621 nit: reducers

  for the last MapReduce task for order by
  ql/src/java/org/apache/hadoop/hive/ql/exec/ apache header
  ql/src/test/queries/clientpositive/orderby_query_bucketing.q:3 can you perform explain extended
  I think, it also shows the number of reducers.
  ql/src/test/queries/clientpositive/orderby_query_bucketing.q:3 Might be easier to create
a tmp table with 10 rows initially to reduce the number of results.
  ql/src/java/org/apache/hadoop/hive/ql/exec/ Add some comments - it would
be good to have a lot of examples.
  ql/src/java/org/apache/hadoop/hive/ql/parse/ What happens if it
is -1 ?

  Shouldn't useBucketingForOrderBy be false ?


To: JIRA, navis
Cc: njain

> Support using multiple reducer for fetching order by results
> ------------------------------------------------------------
>                 Key: HIVE-3972
>                 URL:
>             Project: Hive
>          Issue Type: Improvement
>          Components: Query Processor
>            Reporter: Navis
>            Assignee: Navis
>            Priority: Minor
>         Attachments: HIVE-3972.D8349.1.patch, HIVE-3972.D8349.2.patch
> Queries for fetching results which have lastly "order by" clause make final MR run with
single reducer, which can be too much. For example, 
> {code}
> select value, sum(key) as sum from src group by value order by sum;
> {code}
> If number of reducer is reasonable, multiple result files could be merged into single
sorted stream in the fetcher level.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message