Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id C9380200B13 for ; Wed, 15 Jun 2016 12:30:30 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id C7ABD160A4D; Wed, 15 Jun 2016 10:30:30 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 1B1B7160A4C for ; Wed, 15 Jun 2016 12:30:29 +0200 (CEST) Received: (qmail 21328 invoked by uid 500); 15 Jun 2016 10:30:24 -0000 Mailing-List: contact issues-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list issues@hive.apache.org Received: (qmail 21310 invoked by uid 99); 15 Jun 2016 10:30:24 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 15 Jun 2016 10:30:24 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 0D9342C1F60 for ; Wed, 15 Jun 2016 10:30:24 +0000 (UTC) Date: Wed, 15 Jun 2016 10:30:24 +0000 (UTC) From: "Jesus Camacho Rodriguez (JIRA)" To: issues@hive.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Comment Edited] (HIVE-14002) Extend limit propagation to subsequent RS operators MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Wed, 15 Jun 2016 10:30:31 -0000 [ https://issues.apache.org/jira/browse/HIVE-14002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15331503#comment-15331503 ] Jesus Camacho Rodriguez edited comment on HIVE-14002 at 6/15/16 10:29 AM: -------------------------------------------------------------------------- Thanks for reviewing [~ashutoshc]. In fact, _operator.acceptLimitPushdown()_ is used to bail out if the operator does not accept the limit pushdown. This is the same logic that was used for pushing it from the Limit operator, thus I expect that the operators are properly classified and it is sound. Further, if we cannot backtrack a column from the top RS to the bottom RS, we bail out too (in fact, the new _checkKeys_ method checks that). was (Author: jcamachorodriguez): Thanks for reviewing [~ashutoshc]. In fact, _operator.acceptLimitPushdown()_ is used to bail out if the operator does not accept the limit pushdown. This is the same logic that was used for pushing it from the Limit operator, thus I would expect that the operators are properly classified and it is sound. Further, if we cannot backtrack a column from the top RS to the bottom RS, we bail out too (in fact, the new _checkKeys_ method checks that). > Extend limit propagation to subsequent RS operators > --------------------------------------------------- > > Key: HIVE-14002 > URL: https://issues.apache.org/jira/browse/HIVE-14002 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer > Affects Versions: 2.2.0 > Reporter: Nita Dembla > Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-14002.01.patch, HIVE-14002.patch > > > In some occasions, for instance when RS dedup does not kick in, it is useful to propagate the limit to subsequent RS operators, as this will reduce intermediary results and impact performance. This issue covers that extension. -- This message was sent by Atlassian JIRA (v6.3.4#6332)