phoenix-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-4820) Optimize OrderBy for ClientAggregatePlan
Date Thu, 03 Jan 2019 11:19:00 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-4820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16732928#comment-16732928
] 

Hadoop QA commented on PHOENIX-4820:
------------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12953587/PHOENIX-4820_v3-4.x-HBase-1.3.patch
  against 4.x-HBase-1.3 branch at commit 3671097af252fd15f6481f4b2e7090b10cd0ff14.
  ATTACHMENT ID: 12953587

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 6 new or modified
tests.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:red}-1 release audit{color}.  The applied patch generated 1 release audit warnings
(more than the master's current 0 warnings).

    {color:red}-1 lineLengths{color}.  The patch introduces the following lines longer than
100:
    +    private void doTestOrderByOptimizeForClientAggregatePlanBug4820(boolean desc ,boolean
salted) throws Exception {
+            conn.createStatement().execute("UPSERT INTO "+tableName+" VALUES ('a11','a12','a13','a14','a15')");
+            conn.createStatement().execute("UPSERT INTO "+tableName+" VALUES ('a21','a22','a23','a24','a25')");
+            conn.createStatement().execute("UPSERT INTO "+tableName+" VALUES ('a31','a32','a33','a34','a35')");
+            conn.createStatement().execute("UPSERT INTO "+tableName+" VALUES ('b11','b12','b13','b14','b15')");
+            conn.createStatement().execute("UPSERT INTO "+tableName+" VALUES ('b21','b22','b23','b24','b25')");
+            conn.createStatement().execute("UPSERT INTO "+tableName+" VALUES ('b31','b32','b33','b34','b35')");
+                  "from (select pk1 ak1,pk2 ak2,pk3 ak3, substr(v1,1,1) av1,substr(v2,1,1)
av2 from "+tableName+" order by pk2,pk3 limit 10) a "+
+                  "from (select pk1 ak1,pk2 ak2,pk3 ak3, substr(v1,1,1) av1,substr(v2,1,1)
av2 from "+tableName+" order by pk2,pk3 limit 10) a "+
+                  "from (select pk1 ak1,pk2 ak2,pk3 ak3,substr(v1,1,1) av1,substr(v2,1,1)
av2 from "+tableName+" order by pk2,pk3 limit 10) a "+

     {color:red}-1 core tests{color}.  The patch failed these unit tests:
     ./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SkipScanAfterManualSplitIT

Test results: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2230//testReport/
Release audit warnings: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2230//artifact/patchprocess/patchReleaseAuditWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2230//console

This message is automatically generated.

> Optimize OrderBy for ClientAggregatePlan
> ----------------------------------------
>
>                 Key: PHOENIX-4820
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-4820
>             Project: Phoenix
>          Issue Type: Improvement
>    Affects Versions: 4.14.0
>            Reporter: chenglei
>            Assignee: chenglei
>            Priority: Major
>             Fix For: 4.15.0
>
>         Attachments: PHOENIX-4820-4.x-HBase-1.3.patch, PHOENIX-4820.diff, PHOENIX-4820_v4-4.x-HBase-1.3.patch
>
>
> Given a table
> {code}
>   create table test ( 
>        pk1 varchar not null , 
>        pk2 varchar not null, 
>        pk3 varchar not null,
>         v1 varchar, 
>         v2 varchar, 
>         CONSTRAINT TEST_PK PRIMARY KEY ( 
>               pk1,
>               pk2,
>               pk3 ))
> {code}
> for following sql :
> {code}
>      select a.ak3 
>      from (select substr(pk1,1,1) ak1,substr(pk2,1,1) ak2,substr(pk3,1,1) ak3,substr(v1,1,1)
av1,substr(v2,1,1) av2 from test order by pk2,pk3 limit 10) a  group by a.ak3,a.av1 order
by a.ak3,a.av1
> {code}
> Intuitively, the above OrderBy statement {{order by a.ak3,a.av1}} should be compiled
out because it match the group by statement, but in fact it is not.
> The problem is caused by the {{QueryCompiler.compileSingleQuery}} and {{QueryCompiler.compileSingleFlatQuery}},for
 {{QueryCompiler.compileSingleQuery}} method,because the inner query has order by, so in line
520, local variable {{isInRowKeyOrder}} is false:
> {code}
> 519        context.setCurrentTable(tableRef);
> 520        boolean isInRowKeyOrder = innerPlan.getGroupBy() == GroupBy.EMPTY_GROUP_BY
&& innerPlan.getOrderBy() == OrderBy.EMPTY_ORDER_BY;
> {code}
> In {{QueryCompiler.compileSingleFlatQuery}},when {{OrderByCompiler.compile}} method is
invoked, the last parameter {{isInRowKeyOrder}} is false:
> {code}
> 562        OrderBy orderBy = OrderByCompiler.compile(context, select, groupBy, limit,
offset, projector,
> 563                groupBy == GroupBy.EMPTY_GROUP_BY ? innerPlanTupleProjector : null,
isInRowKeyOrder);
> {code}
> So in following line 156 for  {{OrderByCompiler.compile}},even though the {{tracker.isOrderPreserving}}
is true, the OrderBy  statement could not be compiled out.
> {code}
> 156       if (isInRowKeyOrder && tracker.isOrderPreserving()) {
> {code}      
> In my opinion, with GroupBy, in following line 563 for {{QueryCompiler.compileSingleFlatQuery}}
method, when we call 
> {{OrderByCompiler.compile}} method, we no need to conside the {{isInRowKeyOrder}},  just
like the previous parameter  {{tupleProjector}} does.
> {code}
>  562        OrderBy orderBy = OrderByCompiler.compile(context, select, groupBy, limit,
offset, projector,
>  563                groupBy == GroupBy.EMPTY_GROUP_BY ? innerPlanTupleProjector : null,
isInRowKeyOrder);
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message