drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mehant Baid (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-1549) Support partition pruning for simple IN filters
Date Mon, 20 Oct 2014 22:44:33 GMT

    [ https://issues.apache.org/jira/browse/DRILL-1549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14177583#comment-14177583
] 

Mehant Baid commented on DRILL-1549:
------------------------------------

+1

> Support partition pruning for simple IN filters
> -----------------------------------------------
>
>                 Key: DRILL-1549
>                 URL: https://issues.apache.org/jira/browse/DRILL-1549
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Query Planning & Optimization
>    Affects Versions: 0.6.0
>            Reporter: Aman Sinha
>            Assignee: Aman Sinha
>             Fix For: 0.7.0
>
>         Attachments: 0001-DRILL-1549-Partition-pruning-for-simple-IN-filters.patch
>
>
> Queries containing directory filters of the form 'dir0 IN (1, 2)' currently don't get
considered for partition pruning due to a bug inside the visitor of DirPathBuilder.  This
JIRA is to address the issue for basic IN filters, which actually is converted to an OR condition
by the optimizer (dir0 = 1 OR dir1 = 2).  Note that this JIRA does not cover the more general
IN filters which are combined with other ANDs etc.
> To illustrate, here's the restricted case that we want to fix here: (the explain shows
that the scan is reading all directories, not just 1995, 1996): 
> explain plan for select * from dfs.`/Users/asinha/data/multilevel/parquet` where dir0
in (1995, 1996);
> ...
>  "graph" : [ {
>     "pop" : "parquet-scan",
>     "@id" : 4,
>     "entries" : [ {
>       "path" : "file:/Users/asinha/data/multilevel/parquet/1994/Q1/orders_94_q1.parquet"
>     }, {
>       "path" : "file:/Users/asinha/data/multilevel/parquet/1994/Q2/orders_94_q2.parquet"
>     }, {
>       "path" : "file:/Users/asinha/data/multilevel/parquet/1994/Q3/orders_94_q3.parquet"
>     }, {
>       "path" : "file:/Users/asinha/data/multilevel/parquet/1994/Q4/orders_94_q4.parquet"
>     }, {
>       "path" : "file:/Users/asinha/data/multilevel/parquet/1995/Q1/orders_95_q1.parquet"
>     }, {
>       "path" : "file:/Users/asinha/data/multilevel/parquet/1995/Q2/orders_95_q2.parquet"
>     }, {
>       "path" : "file:/Users/asinha/data/multilevel/parquet/1995/Q3/orders_95_q3.parquet"
>     }, {
>       "path" : "file:/Users/asinha/data/multilevel/parquet/1995/Q4/orders_95_q4.parquet"
>     }, {
>       "path" : "file:/Users/asinha/data/multilevel/parquet/1996/Q1/orders_96_q1.parquet"
>     }, {
>       "path" : "file:/Users/asinha/data/multilevel/parquet/1996/Q2/orders_96_q2.parquet"
>     }, {
>       "path" : "file:/Users/asinha/data/multilevel/parquet/1996/Q3/orders_96_q3.parquet"
>     }, {
>       "path" : "file:/Users/asinha/data/multilevel/parquet/1996/Q4/orders_96_q4.parquet"
>     } ],



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message