hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phabricator (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-3550) Early skipping for limit operator at reduce stage
Date Mon, 08 Oct 2012 07:24:04 GMT

     [ https://issues.apache.org/jira/browse/HIVE-3550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Phabricator updated HIVE-3550:
------------------------------

    Attachment: HIVE-3550.D5943.1.patch

navis requested code review of "HIVE-3550 [jira] Early skipping for limit operator at reduce
stage".
Reviewers: JIRA

  DPAL-1908 Early skipping for limit operator at reduce stage

  Queries with limit clause whose final result is from reduce stage could not be early exited
like map stage. But it seemed possible to just skip remaining lines. It can be accomplished
by adding just two lines. Is there any reason we should not to do that?

  select key from src order by key limit 10;
  1. current
  ExtractOperator: 3 forwarded 500 rows
  LimitOperator: 4 forwarded 10 rows

  2. early skipping
  ExtractOperator: 3 forwarded 12 rows
  LimitOperator: 4 forwarded 10 rows

TEST PLAN
  EMPTY

REVISION DETAIL
  https://reviews.facebook.net/D5943

AFFECTED FILES
  ql/src/java/org/apache/hadoop/hive/ql/exec/ExecReducer.java

MANAGE HERALD DIFFERENTIAL RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/14115/

To: JIRA, navis

                
> Early skipping for limit operator at reduce stage
> -------------------------------------------------
>
>                 Key: HIVE-3550
>                 URL: https://issues.apache.org/jira/browse/HIVE-3550
>             Project: Hive
>          Issue Type: Improvement
>          Components: Query Processor
>            Reporter: Navis
>            Assignee: Navis
>            Priority: Trivial
>         Attachments: HIVE-3550.D5943.1.patch
>
>
> Queries with limit clause whose final result is from reduce stage could not be early
exited like map stage. But it seemed possible to just skip remaining lines. It can be accomplished
by adding just two lines. Is there any reason we should not to do that?
> {noformat}
> select key from src order by key limit 10;
> 1. current
> ExtractOperator: 3 forwarded 500 rows
> LimitOperator: 4 forwarded 10 rows
> 2. early skipping
> ExtractOperator: 3 forwarded 12 rows
> LimitOperator: 4 forwarded 10 rows
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message