drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-6099) Drill does not push limit past project (flatten) if it cannot be pushed into scan
Date Thu, 01 Mar 2018 22:04:00 GMT

    [ https://issues.apache.org/jira/browse/DRILL-6099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16382731#comment-16382731
] 

ASF GitHub Bot commented on DRILL-6099:
---------------------------------------

Github user gparai commented on a diff in the pull request:

    https://github.com/apache/drill/pull/1096#discussion_r171708439
  
    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/planner/common/DrillRelOptUtil.java
---
    @@ -224,4 +226,64 @@ public Void visitInputRef(RexInputRef inputRef) {
         }
       }
     
    +  public static boolean isLimit0(RexNode fetch) {
    +    if (fetch != null && fetch.isA(SqlKind.LITERAL)) {
    +      RexLiteral l = (RexLiteral) fetch;
    +      switch (l.getTypeName()) {
    +        case BIGINT:
    +        case INTEGER:
    +        case DECIMAL:
    +          if (((long) l.getValue2()) == 0) {
    +            return true;
    +          }
    +      }
    +    }
    +    return false;
    +  }
    +
    +  public static boolean isProjectOutputRowcountUnknown(RelNode project) {
    +    assert project instanceof Project : "Rel is NOT an instance of project!";
    +    try {
    +      RexVisitor<Void> visitor =
    +          new RexVisitorImpl<Void>(true) {
    +            public Void visitCall(RexCall call) {
    +              if ("flatten".equals(call.getOperator().getName().toLowerCase())) {
    +                throw new Util.FoundOne(call); /* throw exception to interrupt tree walk
(this is similar to
    +                                              other utility methods in RexUtil.java */
    +              }
    +              return super.visitCall(call);
    +            }
    +          };
    +      for (RexNode rex : ((Project) project).getProjects()) {
    +        rex.accept(visitor);
    +      }
    +    } catch (Util.FoundOne e) {
    +      Util.swallow(e, null);
    +      return true;
    +    }
    +    return false;
    +  }
    +
    +  public static boolean isProjectOutputSchemaUnknown(RelNode project) {
    --- End diff --
    
    Done


> Drill does not push limit past project (flatten) if it cannot be pushed into scan
> ---------------------------------------------------------------------------------
>
>                 Key: DRILL-6099
>                 URL: https://issues.apache.org/jira/browse/DRILL-6099
>             Project: Apache Drill
>          Issue Type: Bug
>    Affects Versions: 1.12.0
>            Reporter: Gautam Kumar Parai
>            Assignee: Gautam Kumar Parai
>            Priority: Major
>             Fix For: 1.13.0
>
>
> It would be useful to have pushdown occur past flatten(project). Here is an example to
illustrate the issue:
> {{explain plan without implementation for }}{{select name, flatten(categories) as category
from dfs.`/tmp/t_json_20` LIMIT 1;}}
> {{DrillScreenRel}}{{  }}
> {{  DrillLimitRel(fetch=[1])}}{{    }}
> {{    DrillProjectRel(name=[$0], category=[FLATTEN($1)])}}
> {{      DrillScanRel(table=[[dfs, /tmp/t_json_20]], groupscan=[EasyGroupScan [selectionRoot=maprfs:/tmp/t_json_20,
numFiles=1, columns=[`name`, `categories`], files=[maprfs:///tmp/t_json_20/0_0_0.json]]])}}
> ================================================================= 
> Content of 0_0_0.json
> =================================================================
> {
>   "name" : "Eric Goldberg, MD",
>   "categories" : [ "Doctors", "Health & Medical" ]
> } {
>   "name" : "Pine Cone Restaurant",
>   "categories" : [ "Restaurants" ]
> } {
>   "name" : "Deforest Family Restaurant",
>   "categories" : [ "American (Traditional)", "Restaurants" ]
> } {
>   "name" : "Culver's",
>   "categories" : [ "Food", "Ice Cream & Frozen Yogurt", "Fast Food", "Restaurants"
]
> } {
>   "name" : "Chang Jiang Chinese Kitchen",
>   "categories" : [ "Chinese", "Restaurants" ]
> } 
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message