drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-6318) Push down limit past flatten is incorrect
Date Tue, 10 Apr 2018 19:22:00 GMT

    [ https://issues.apache.org/jira/browse/DRILL-6318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16432803#comment-16432803
] 

ASF GitHub Bot commented on DRILL-6318:
---------------------------------------

Github user gparai commented on a diff in the pull request:

    https://github.com/apache/drill/pull/1204#discussion_r180539313
  
    --- Diff: exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/flatten/TestFlattenPlanning.java
---
    @@ -66,8 +66,11 @@ public void testPushFilterPastProjectWithFlattenNeg() throws Exception
{
       @Test // DRILL-6099 : push limit past flatten(project)
       public void testLimitPushdownPastFlatten() throws Exception {
         final String query = "select rownum, flatten(complex) comp from cp.`store/json/test_flatten_mappify2.json`
limit 1";
    -    final String[] expectedPatterns = {".*Limit\\(fetch=\\[1\\]\\).*",".*Flatten.*",".*Limit\\(fetch=\\[1\\]\\).*"};
    -    final String[] excludedPatterns = null;
    +    //DRILL-6318 : limit should not push past flatten(project)
    +    //P.S. Where was an error in this pattern. Even then Limit missing after Flatten
it matches to plan
    --- End diff --
    
    Remove the P.S. and commented out pattern.


> Push down limit past flatten is incorrect
> -----------------------------------------
>
>                 Key: DRILL-6318
>                 URL: https://issues.apache.org/jira/browse/DRILL-6318
>             Project: Apache Drill
>          Issue Type: Bug
>    Affects Versions: 1.13.0, 1.14.0
>         Environment: Ubuntu Mate 18.04
> Apache Drill 1.14.0-SNAPSHOT
>            Reporter: Oleg Zinoviev
>            Priority: Major
>         Attachments: sample.json
>
>
> Sample data: [^sample.json]
> Result of
> {code:sql}
> select flatten(data) as d from dfs.root.`sample.json`
> {code}
> is
> ||d||
> |Honored Boy Scout|
> |Yawning Wolf|
> |Closed Queen|
> |Innocent Volunteer|
> |Junior Wing|
> |Lame Mantis|
> |Old Master|
> |Numb Pawn|
> |Hollow Guardian|
> |Twin Hurricane|
> |Helpless Avalange|
> Let's try to get first 3 rows:
> {code:sql}
> select flatten(data) as d from dfs.root.`sample.json` limit 3
> {code}
> Result has only 2 rows:
> ||d||
> |Honored Boy Scout|
> |Yawning Wolf|
> *Reason:* Limit was pushed down below flatten and only 3 top rows from json was selected.
In this 3 rows only 2nd has items in "data" field.
> Let's try to get 3 rows from the middle:
> {code:sql}
> select flatten(data) as d from dfs.root.`sample.json` limit 3 offset 5
> {code}
> Result is empty.
> *Reason:* Limit and offset was pushed down below flatten and only 6, 7 and 8 row from
json was selected. This 3 rows contains only 3 items in "data" field. After flatten limit
and offset applies second time and reject all select items.
> Error in org/apache/drill/exec/planner/logical/DrillPushLimitToScanRule.java



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message