hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phabricator (JIRA)" <>
Subject [jira] [Commented] (HIVE-3562) Some limit can be pushed down to map stage
Date Thu, 06 Dec 2012 15:09:10 GMT


Phabricator commented on HIVE-3562:

njain has commented on the revision "HIVE-3562 [jira] Some limit can be pushed down to map

  As per jira comments, can you add a new parameter for the limit on k ?

  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ Instead of
checking for instanceof, can you add a method in Operator() - something like
  canLimitBePushed() with a lot of comments, and then the above operators can have this to
  ql/src/java/org/apache/hadoop/hive/ql/exec/  Can you add a heap
based implementation as suggested in jira ?
  ql/src/java/org/apache/hadoop/hive/ql/exec/ Can you add lot of
comments here - when is this set ?
  what queries will it benefit etc. ?
  common/src/java/org/apache/hadoop/hive/conf/ add these in hive-default.xml.template
  ql/src/test/queries/clientpositive/limit_pushdown.q:7 can you add another positive/negative
test ?

  explain select value, sum(key) from src group by value limit 10;

  The limit should be used in the 2nd MR job if u r inserting into a table.
  ql/src/test/queries/clientpositive/limit_pushdown.q:10 For the 2 negative queries, can you
insert into a table also -
  then the optm. should help


To: JIRA, navis
Cc: njain

> Some limit can be pushed down to map stage
> ------------------------------------------
>                 Key: HIVE-3562
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Navis
>            Assignee: Navis
>            Priority: Trivial
>         Attachments: HIVE-3562.D5967.1.patch
> Queries with limit clause (with reasonable number), for example
> {noformat}
> select * from src order by key limit 10;
> {noformat}
> makes operator tree, 
> But LIMIT can be partially calculated in RS, reducing size of shuffling.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message