hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vikram Dixit K (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-7302) Allow Auto-reducer parallelism to be turned off by a logical optimizer
Date Fri, 27 Jun 2014 07:33:24 GMT

    [ https://issues.apache.org/jira/browse/HIVE-7302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14045662#comment-14045662
] 

Vikram Dixit K commented on HIVE-7302:
--------------------------------------

I discussed this with Gopal today and I think that is the right place to replace it. I tested
this on the bucket map join tests I am currently modifying and getting the right behavior.

Showing this info in the explain extended plan is a good idea. One issue would be if during
execution we actually change the number of reducers in tez, the number shown may not have
any real meaning apart from saying there was an estimation.

> Allow Auto-reducer parallelism to be turned off by a logical optimizer
> ----------------------------------------------------------------------
>
>                 Key: HIVE-7302
>                 URL: https://issues.apache.org/jira/browse/HIVE-7302
>             Project: Hive
>          Issue Type: Bug
>          Components: Tez
>    Affects Versions: 0.14.0
>            Reporter: Gopal V
>            Assignee: Gopal V
>         Attachments: HIVE-7302.1.patch
>
>
> Auto reducer parallelism cannot be used for cases where a custom routing VertexManager
is used.
> Allow a tri-state for ReduceSinkDesc::isAutoParallel to allow "allow, disable, unset"
mechanics.
> The state machine for this setting will now be 
> Allowed transitions
> unset -> allow
> unset -> disable
> allow -> disable
> with no transition case for
> disable -> allow.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message