hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-12462) DPP: DPP optimizers need to run on the TS predicate not FIL
Date Mon, 23 Nov 2015 22:28:11 GMT

    [ https://issues.apache.org/jira/browse/HIVE-12462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15023228#comment-15023228
] 

Sergey Shelukhin commented on HIVE-12462:
-----------------------------------------

Test failures are because of the missing cleanup (currently, the code cleans up TS expr separately
and filter expr as part of processing, if TS expr is used in processing noone cleans up the
filter). 

> DPP: DPP optimizers need to run on the TS predicate not FIL 
> ------------------------------------------------------------
>
>                 Key: HIVE-12462
>                 URL: https://issues.apache.org/jira/browse/HIVE-12462
>             Project: Hive
>          Issue Type: Bug
>          Components: Tez
>    Affects Versions: 2.0.0
>            Reporter: Gopal V
>            Assignee: Gopal V
>            Priority: Critical
>         Attachments: HIVE-12462.1.patch
>
>
> HIVE-11398 + HIVE-11791, the partition-condition-remover became more effective.
> This removes predicates from the FilterExpression which involve partition columns, causing
a miss for dynamic-partition pruning if the DPP relies on FilterDesc.
> The TS desc will have the correct predicate in that condition.
> {code}
> $hdt$_0:$hdt$_1:a
>   TableScan (TS_2)
>     alias: a
>     filterExpr: (((account_id = 22) and year(dt) is not null) and (year(dt)) IN (RS[6]))
(type: boolean)
>     Filter Operator (FIL_20)
>       predicate: ((account_id = 22) and year(dt) is not null) (type: boolean)
>       Select Operator (SEL_4)
>         expressions: dt (type: date)
>         outputColumnNames: _col1
>         Reduce Output Operator (RS_8)
>           key expressions: year(_col1) (type: int)
>           sort order: +
>           Map-reduce partition columns: year(_col1) (type: int)
>           Join Operator (JOIN_9)
>             condition map:
>                  Inner Join 0 to 1
>             keys:
>               0 year(_col1) (type: int)
>               1 year(_col1) (type: int)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message