From dev-return-50980-archive-asf-public=cust-asf.ponee.io@phoenix.apache.org Mon Apr 16 05:50:04 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 039D7180648 for ; Mon, 16 Apr 2018 05:50:03 +0200 (CEST) Received: (qmail 94472 invoked by uid 500); 16 Apr 2018 03:50:02 -0000 Mailing-List: contact dev-help@phoenix.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@phoenix.apache.org Delivered-To: mailing list dev@phoenix.apache.org Received: (qmail 94461 invoked by uid 99); 16 Apr 2018 03:50:02 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 16 Apr 2018 03:50:02 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 41634C64E9 for ; Mon, 16 Apr 2018 03:50:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -109.501 X-Spam-Level: X-Spam-Status: No, score=-109.501 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id dUfWs7HonWsr for ; Mon, 16 Apr 2018 03:50:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 151C15F2C3 for ; Mon, 16 Apr 2018 03:50:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 62559E030E for ; Mon, 16 Apr 2018 03:50:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 2135E21209 for ; Mon, 16 Apr 2018 03:50:00 +0000 (UTC) Date: Mon, 16 Apr 2018 03:50:00 +0000 (UTC) From: "chenglei (JIRA)" To: dev@phoenix.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (PHOENIX-4690) GroupBy expressions should follow the order of PK Columns if GroupBy is orderPreserving MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/PHOENIX-4690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenglei updated PHOENIX-4690: ------------------------------ Affects Version/s: 4.14.0 > GroupBy expressions should follow the order of PK Columns if GroupBy is orderPreserving > --------------------------------------------------------------------------------------- > > Key: PHOENIX-4690 > URL: https://issues.apache.org/jira/browse/PHOENIX-4690 > Project: Phoenix > Issue Type: Bug > Affects Versions: 4.14.0, 4.13.2 > Reporter: chenglei > Priority: Critical > Fix For: 4.14.0 > > Attachments: PHOENIX-4690_v1.patch > > > Given a table : > {code} > create table test ( > pk1 integer not null , > pk2 integer not null, > v integer, > CONSTRAINT TEST_PK PRIMARY KEY (pk1,pk2)) > {code} > and some data: > {code} > +------+------+-----+ > | PK1 | PK2 | V | > +------+------+-----+ > | 1 | 8 | 10 | > | 1 | 9 | 11 | > | 2 | 3 | 13 | > | 2 | 7 | 15 | > | 3 | 2 | 17 | > +------+------+-----+ > {code} > for following sql : > {code} > select pk2,pk1,count(v) from test group by pk2,pk1 order by pk2,pk1 > {code} > the expected result is : > {code} > +------+------+-----------+ > | PK2 | PK1 | COUNT(V) | > +------+------+-----------+ > | 2 | 3 | 1 | > | 3 | 2 | 1 | > | 7 | 2 | 1 | > | 8 | 1 | 1 | > | 9 | 1 | 1 | > +------+------+-----------+ > {code} > but the actual result is : > {code} > +------+------+-----------+ > | PK2 | PK1 | COUNT(V) | > +------+------+-----------+ > | 8 | 1 | 1 | > | 9 | 1 | 1 | > | 3 | 2 | 1 | > | 7 | 2 | 1 | > | 2 | 3 | 1 | > +------+------+-----------+ > {code} > The problem is caused by the {{GroupBy.compile}}, obviously, in line 154, for {{group by pk2,pk1}}, > {{OrderPreservingTracker.isOrderPreserving}} is true by reorder the {{pk2,pk1}} to {{pk1,pk2}} following the order of PK columns {{pk1,pk2}}, but in line 158, for the new GroupBy, the GroupBy.expressions is still {{pk2,pk1}},not the actual {{pk1,pk2}}: > {code} > 141 public GroupBy compile(StatementContext context, TupleProjector tupleProjector) throws SQLException { > 142 boolean isOrderPreserving = this.isOrderPreserving; > 143 int orderPreservingColumnCount = 0; > 144 if (isOrderPreserving) { > 145 OrderPreservingTracker tracker = new OrderPreservingTracker(context, GroupBy.EMPTY_GROUP_BY, Ordering.UNORDERED, expressions.size(), tupleProjector); > 146 for (int i = 0; i < expressions.size(); i++) { > 147 Expression expression = expressions.get(i); > 148 tracker.track(expression); > 149 } > 150 > 151 // This is true if the GROUP BY is composed of only PK columns. We further check here that > 152 // there are no "gaps" in the PK columns positions used (i.e. we start with the first PK > 153 // column and use each subsequent one in PK order). > 154 isOrderPreserving = tracker.isOrderPreserving(); > 155 orderPreservingColumnCount = tracker.getOrderPreservingColumnCount(); > 156 } > 157 if (isOrderPreserving || isUngroupedAggregate) { > 158 return new GroupBy.GroupByBuilder(this).setIsOrderPreserving(isOrderPreserving) > .setOrderPreservingColumnCount(orderPreservingColumnCount).build(); > 159 } > {code} > Then when we compile {{order by pk2,pk1}} in {{OrderByCompiler.compile}}, because {{the GroupBy.isOrderPreserving}} is true, and {{order by pk2,pk1}} is consistent with the GroupBy.expressions {{group by pk2,pk1}} created in above {{GroupBy.compile}} method, so the result of {{OrderByCompiler.compile}} is {{OrderBy.FWD_ROW_KEY_ORDER_BY}}. > But in fact,because the actual GroupBy.expressions is {{group by pk1,pk2}},so we need to execute {{order by pk2,pk1}} after the {{group by pk1,pk2}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)