hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashutosh Chauhan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-8163) With dynamic partition pruning map operator that generates the partition filters is not vectorized
Date Tue, 23 Sep 2014 05:04:34 GMT

    [ https://issues.apache.org/jira/browse/HIVE-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14144347#comment-14144347
] 

Ashutosh Chauhan commented on HIVE-8163:
----------------------------------------

+1

> With dynamic partition pruning map operator that generates the partition filters is not
vectorized
> --------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-8163
>                 URL: https://issues.apache.org/jira/browse/HIVE-8163
>             Project: Hive
>          Issue Type: Bug
>          Components: Tez
>    Affects Versions: 0.14.0
>            Reporter: Mostafa Mokhtar
>            Assignee: Gunther Hagleitner
>            Priority: Minor
>              Labels: performance
>         Attachments: HIVE-8163.1.patch, HIVE-8163.2.patch
>
>
> Vertex used to generate the partition pruning filters is not vectorized.
> Sample from the plan :
> {code}
>     Vertices:
>         Map 1 
>             Map Operator Tree:
>                 TableScan
>                   alias: d3
>                   filterExpr: ((d_quarter_name) IN ('2000Q1', '2000Q2', '2000Q3') and
d_date_sk is not null) (type: boolean)
>                   Statistics: Num rows: 73049 Data size: 81741831 Basic stats: COMPLETE
Column stats: NONE
>                   Filter Operator
>                     predicate: ((d_quarter_name) IN ('2000Q1', '2000Q2', '2000Q3') and
d_date_sk is not null) (type: boolean)
>                     Statistics: Num rows: 18262 Data size: 20435178 Basic stats: COMPLETE
Column stats: NONE
>                     Select Operator
>                       expressions: d_date_sk (type: int)
>                       outputColumnNames: _col0
>                       Statistics: Num rows: 18262 Data size: 20435178 Basic stats: COMPLETE
Column stats: NONE
>                       Reduce Output Operator
>                         key expressions: _col0 (type: int)
>                         sort order: +
>                         Map-reduce partition columns: _col0 (type: int)
>                         Statistics: Num rows: 18262 Data size: 20435178 Basic stats:
COMPLETE Column stats: NONE
>                       Select Operator
>                         expressions: _col0 (type: int)
>                         outputColumnNames: _col0
>                         Statistics: Num rows: 18262 Data size: 20435178 Basic stats:
COMPLETE Column stats: NONE
>                         Group By Operator
>                           keys: _col0 (type: int)
>                           mode: hash
>                           outputColumnNames: _col0
>                           Statistics: Num rows: 18262 Data size: 20435178 Basic stats:
COMPLETE Column stats: NONE
>                           Dynamic Partitioning Event Operator
>                             Target Input: catalog_sales
>                             Partition key expr: cs_sold_date_sk
>                             Statistics: Num rows: 18262 Data size: 20435178 Basic stats:
COMPLETE Column stats: NONE
>                             Target column: cs_sold_date_sk
>                             Target Vertex: Map 3
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message