drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jacques Nadeau (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (DRILL-1571) Partition pruning pushes filter into scan but does not eliminate the filter
Date Sun, 04 Jan 2015 21:29:38 GMT

     [ https://issues.apache.org/jira/browse/DRILL-1571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jacques Nadeau updated DRILL-1571:
----------------------------------
    Fix Version/s:     (was: 0.8.0)
                   0.9.0
         Priority: Minor  (was: Major)

> Partition pruning pushes filter into scan but does not eliminate the filter
> ---------------------------------------------------------------------------
>
>                 Key: DRILL-1571
>                 URL: https://issues.apache.org/jira/browse/DRILL-1571
>             Project: Apache Drill
>          Issue Type: Improvement
>    Affects Versions: 0.6.0
>            Reporter: Mehant Baid
>            Assignee: Mehant Baid
>            Priority: Minor
>             Fix For: 0.9.0
>
>
> select * from dfs.`/tmp/data` where (dir0 = 1997 and dir1 = 1) or (dir0 = 1998 and dir1
= 3)
> Here is the plan for the above query:
> 00-00    Screen
> 00-01      Project(*=[$0])
> 00-02        SelectionVectorRemover
> 00-03          Filter(condition=[OR(AND(=(CAST($1):INTEGER, 1997), =(CAST($2):INTEGER,
1)), AND(=(CAST($1):INTEGER, 1998), =(CAST($2):INTEGER, 3)))])
> 00-04            Scan(groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath [path=/tmp/data/1997/1/region.parquet],
ReadEntryWithPath [path=/tmp/data/1998/3/region.parquet]], selectionRoot=/tmp/data, numFiles=2,
columns=[SchemaPath [`*`]]]])
> From the plan its clear that we are pushing the partition filters into the scan however
even though all the filter conditions are partition conditions we are not eliminating the
filter entirely. 
> Even though we prune and avoid scanning the unwanted directories it would be good to
eliminate the filter altogether in such simple cases.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message