drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rahul Challapalli (JIRA)" <j...@apache.org>
Subject [jira] [Closed] (DRILL-4071) Partition pruning fails when a Coalesce() function appears with partition filter
Date Mon, 14 Dec 2015 22:25:46 GMT

     [ https://issues.apache.org/jira/browse/DRILL-4071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Rahul Challapalli closed DRILL-4071.
------------------------------------

Verified the fix and added a testcase

> Partition pruning fails when a Coalesce() function appears with partition filter
> --------------------------------------------------------------------------------
>
>                 Key: DRILL-4071
>                 URL: https://issues.apache.org/jira/browse/DRILL-4071
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Query Planning & Optimization
>            Reporter: Aman Sinha
>            Assignee: Mehant Baid
>             Fix For: 1.3.0
>
>         Attachments: DRILL-4071.patch
>
>
> Pruning fails for this query: 
> {code}
> 0: jdbc:drill:zk=local> explain plan for select count(*) from dfs.`/Users/asinha/data/multilevel/parquet`
where dir0 = 1994 and coalesce(o_clerk, 'Clerk') = '';
> +------+------+
> | text | json |
> +------+------+
> | 00-00    Screen
> 00-01      Project(EXPR$0=[$0])
> 00-02        StreamAgg(group=[{}], EXPR$0=[COUNT()])
> 00-03          Project($f0=[0])
> 00-04            SelectionVectorRemover
> 00-05              Filter(condition=[AND(=($0, 1994), =(CASE(IS NOT NULL($1), $1, 'Clerk'),
''))])
> 00-06                Scan(groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath [path=file:/Users/asinha/data/multilevel/parquet/1994/Q1/orders_94_q1.parquet],
ReadEntryWithPath [path=file:/Users/asinha/data/multilevel/parquet/1994/Q2/orders_94_q2.parquet],
ReadEntryWithPath [path=file:/Users/asinha/data/multilevel/parquet/1994/Q3/orders_94_q3.parquet],
ReadEntryWithPath [path=file:/Users/asinha/data/multilevel/parquet/1994/Q4/orders_94_q4.parquet],
ReadEntryWithPath [path=file:/Users/asinha/data/multilevel/parquet/1995/Q1/orders_95_q1.parquet],
ReadEntryWithPath [path=file:/Users/asinha/data/multilevel/parquet/1995/Q2/orders_95_q2.parquet],
ReadEntryWithPath [path=file:/Users/asinha/data/multilevel/parquet/1995/Q3/orders_95_q3.parquet],
ReadEntryWithPath [path=file:/Users/asinha/data/multilevel/parquet/1995/Q4/orders_95_q4.parquet],
ReadEntryWithPath [path=file:/Users/asinha/data/multilevel/parquet/1996/Q1/orders_96_q1.parquet],
ReadEntryWithPath [path=file:/Users/asinha/data/multilevel/parquet/1996/Q2/orders_96_q2.parquet],
ReadEntryWithPath [path=file:/Users/asinha/data/multilevel/parquet/1996/Q3/orders_96_q3.parquet],
ReadEntryWithPath [path=file:/Users/asinha/data/multilevel/parquet/1996/Q4/orders_96_q4.parquet]],
selectionRoot=file:/Users/asinha/data/multilevel/parquet, numFiles=12, usedMetadataFile=false,
columns=[`dir0`, `o_clerk`]]])
> {code}
> The log indicates no partition filters were found: 
> {code}
> ...
> o.a.d.e.p.l.partition.PruneScanRule - No conditions were found eligible for partition
pruning.Total pruning elapsed time: 0 ms
> {code}
> A preliminary analysis indicates that since the Coalesce gets converted to a CASE(IS
NOT NULL....) expression, the filter analysis does not correctly process the full expression
tree.  At one point in {{FindPartitionConditions.analyzeCall()}} I saw the operandStack had
3 elements in it: [NO_PUSH, NO_PUSH, PUSH] which seemed strange since I would expect even
number of elements. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message