drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rahul Challapalli (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-1610) Partition pruning not scanning all the required partitions when we have hierarchical partitions
Date Fri, 14 Nov 2014 02:33:35 GMT

    [ https://issues.apache.org/jira/browse/DRILL-1610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14211732#comment-14211732
] 

Rahul Challapalli commented on DRILL-1610:
------------------------------------------

Tested this with the latest build (git.commit.id.abbrev=fd54e71). However drill now seems
to be scanning all the partitions, effectively ignoring the filters while scanning. 

> Partition pruning not scanning all the required partitions when we have hierarchical
partitions
> -----------------------------------------------------------------------------------------------
>
>                 Key: DRILL-1610
>                 URL: https://issues.apache.org/jira/browse/DRILL-1610
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Query Planning & Optimization
>    Affects Versions: 0.7.0
>            Reporter: Rahul Challapalli
>         Attachments: DRILL-1610.patch, explain_plan_dfs.txt, explain_plan_hive.txt, hive.ddl,
lineitem_hierarchical.zip
>
>
> git.commit.id.abbrev=6dca24a
> My data is organized in such a way that we have multiple year partitions and each year
partition has 12 month partitions. 
> Now the below query should look for all the records 
>   1. all months under 1993 
>   2. under the month 'dec' for all the years
> However drill seems to be ignoring the second case. This is happening with hive and dfs.
> Below are the queries :
> DFS :
> {code}
> explain plan for select 
>   l_orderkey, 
>   l_partkey, 
>   l_quantity, 
>   l_shipdate, 
>   l_shipinstruct 
> from hive.lineitem_hierarchical where (`year` IN (1993) and l_orderkey>29600) or `month`
IN ('dec');
> {code}
> Hive :
> {code}
> explain plan for select columns[0],
>   columns[1],
>   columns[4],
>   columns[10],
>   columns[13]
> from `/lineitem_hierarchical` where (dir0=1993 and columns[0]>29600) or dir1='dec';
> {code}
> Attached the sample data, hive ddl, and the explain plan outputs



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message