hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jesus Camacho Rodriguez (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HIVE-14002) Extend limit propagation to subsequent RS operators
Date Wed, 15 Jun 2016 10:29:09 GMT

    [ https://issues.apache.org/jira/browse/HIVE-14002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15331503#comment-15331503
] 

Jesus Camacho Rodriguez edited comment on HIVE-14002 at 6/15/16 10:29 AM:
--------------------------------------------------------------------------

Thanks for reviewing [~ashutoshc]. In fact, _operator.acceptLimitPushdown()_ is used to bail
out if the operator does not accept the limit pushdown. This is the same logic that was used
for pushing it from the Limit operator, thus I would expect that the operators are properly
classified and it is sound.

Further, if we cannot backtrack a column from the top RS to the bottom RS, we bail out too
(in fact, the new _checkKeys_ method checks that).


was (Author: jcamachorodriguez):
Thanks for reviewing [~ashutoshc]. In fact, _operator.acceptLimitPushdown()_ is used to bail
out if the operator does not accept the limit pushdown. This is the same logic that was used
for pushing it from the Limit operator, thus I would expect that it is sound.

Further, if we cannot backtrack a column from the top RS to the bottom RS, we bail out too
(in fact, the new _checkKeys_ method does that).

> Extend limit propagation to subsequent RS operators
> ---------------------------------------------------
>
>                 Key: HIVE-14002
>                 URL: https://issues.apache.org/jira/browse/HIVE-14002
>             Project: Hive
>          Issue Type: Improvement
>          Components: Physical Optimizer
>    Affects Versions: 2.2.0
>            Reporter: Nita Dembla
>            Assignee: Jesus Camacho Rodriguez
>         Attachments: HIVE-14002.01.patch, HIVE-14002.patch
>
>
> In some occasions, for instance when RS dedup does not kick in, it is useful to propagate
the limit to subsequent RS operators, as this will reduce intermediary results and impact
performance. This issue covers that extension.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message