hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rajesh Balamohan (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HIVE-15942) Q22 does not get vectorized due to grouping set evaluations
Date Thu, 04 May 2017 06:13:04 GMT

     [ https://issues.apache.org/jira/browse/HIVE-15942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Rajesh Balamohan resolved HIVE-15942.
-------------------------------------
    Resolution: Fixed

Checked with apache master. {{Map 1}} gets vectorized now with Q22. Marking this ticket as
fixed.

> Q22 does not get vectorized due to grouping set evaluations
> -----------------------------------------------------------
>
>                 Key: HIVE-15942
>                 URL: https://issues.apache.org/jira/browse/HIVE-15942
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Rajesh Balamohan
>         Attachments: query_plan_q22_HIVE-15942.txt
>
>
> Env: apache hive master with LLAP + tez master 
> Query: q22 @ tpcds 10 TB scale
> Map-1 does not get vectorized. Hive logs point out that group set evaluation is preventing
it from vectorization
> {noformat}
> 2017-02-16T07:10:06,074  INFO [c9d014ef-5a60-4ef1-b7a8-5209da679ebf main] physical.Vectorizer:
ReduceWorkVectorizationNodeProcessor process reduceColumnNames [VALUE._col0, VALUE._col1,
VALUE._col2]
> 2017-02-16T07:10:06,074  INFO [c9d014ef-5a60-4ef1-b7a8-5209da679ebf main] physical.Vectorizer:
ReduceWorkVectorizationNodeProcessor process operator GBY using vectorization contextContext
name __Reduce_Shuffle__, level 0, sorted projectionColumnMap {0=VALUE._col0, 1=VALUE._col1,
2=VALUE._col2}, scratchColumnTypeNames []
> 2017-02-16T07:10:06,074  INFO [c9d014ef-5a60-4ef1-b7a8-5209da679ebf main] physical.Vectorizer:
ReduceWorkVectorizationNodeProcessor process going to walk the operator stack to get vectorization
context for RS
> 2017-02-16T07:10:06,075  INFO [c9d014ef-5a60-4ef1-b7a8-5209da679ebf main] physical.Vectorizer:
walkStackToFindVectorizationContext GBY has new vectorization context Context name GBY, level
0, sorted projectionColumnMap {0=_col0, 1=_col1, 2=_col2}, scratchColumnTypeNames []
> 2017-02-16T07:10:06,075  INFO [c9d014ef-5a60-4ef1-b7a8-5209da679ebf main] physical.Vectorizer:
ReduceWorkVectorizationNodeProcessor process operator RS using vectorization contextContext
name GBY, level 0, sorted projectionColumnMap {0=_col0, 1=_col1, 2=_col2}, scratchColumnTypeNames
[]
> 2017-02-16T07:10:06,075  INFO [c9d014ef-5a60-4ef1-b7a8-5209da679ebf main] physical.Vectorizer:
Validating MapWork...
> 2017-02-16T07:10:06,084  INFO [c9d014ef-5a60-4ef1-b7a8-5209da679ebf main] physical.Vectorizer:
Cannot vectorize: GROUPBY operator: Grouping sets not supported
> 2017-02-16T07:10:06,084  INFO [c9d014ef-5a60-4ef1-b7a8-5209da679ebf main] physical.Vectorizer:
Validating ReduceWork...
> 2017-02-16T07:10:06,084  INFO [c9d014ef-5a60-4ef1-b7a8-5209da679ebf main] physical.Vectorizer:
Cannot vectorize: GROUPBY operator: Pruning grouping set id not supported
> 2017-02-16T07:10:06,085  INFO [c9d014ef-5a60-4ef1-b7a8-5209da679ebf main] physical.Vectorizer:
Validating ReduceWork...
> 2017-02-16T07:10:06,086  INFO [c9d014ef-5a60-4ef1-b7a8-5209da679ebf main] physical.Vectorizer:
Vectorizing ReduceWork...
> 2017-02-16T07:10:06,086  INFO [c9d014ef-5a60-4ef1-b7a8-5209da679ebf main] physical.Vectorizer:
vectorizeReduceWork reducer Operator: SEL...
> 2017-02-16T07:10:06,086  INFO [c9d014ef-5a60-4ef1-b7a8-5209da679ebf main] physical.Vectorizer:
ReduceWorkVectorizationNodeProcessor process reduceColumnNames [KEY.reducesinkkey0, KEY.reducesinkkey1,
KEY.reducesinkkey2, KEY.reducesinkkey3, KEY.reducesinkkey4]
> 2017-02-16T07:10:06,086  INFO [c9d014ef-5a60-4ef1-b7a8-5209da679ebf main] physical.Vectorizer:
ReduceWorkVectorizationNodeProcessor process operator SEL using vectorization contextContext
name __Reduce_Shuffle__, level 0, sorted projectionColumnMap {0=KEY.reducesinkkey0, 1=KEY.reducesinkkey1,
2=KEY.reducesinkkey2, 3=KEY.reducesinkkey3, 4=KEY.reducesinkkey4}, scratchColumnTypeNames
[]
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message