hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gunther Hagleitner (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-5095) Hive needs new operator walker for parallelization/optimization for tez
Date Thu, 29 Aug 2013 23:33:52 GMT

    [ https://issues.apache.org/jira/browse/HIVE-5095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13754198#comment-13754198
] 

Gunther Hagleitner commented on HIVE-5095:
------------------------------------------

Committed .1 to branch. Thanks!
                
> Hive needs new operator walker for parallelization/optimization for tez
> -----------------------------------------------------------------------
>
>                 Key: HIVE-5095
>                 URL: https://issues.apache.org/jira/browse/HIVE-5095
>             Project: Hive
>          Issue Type: Bug
>          Components: Tez
>    Affects Versions: tez-branch
>            Reporter: Vikram Dixit K
>            Assignee: Vikram Dixit K
>             Fix For: tez-branch
>
>         Attachments: HIVE-5095.1.patch, HIVE-5095.WIP.patch.txt
>
>
> For tez to compute the number of reducers, we should be walking the operator tree in
a topological fashion so that the reducers down the tree get the estimate from all parents.
However, the current walkers in hive only walk the operator tree in a depth-first fashion.
We need to add a new walker for the topological walk. Also, since information about the parent
operators needs to be propagated on a per parent basis, we need to retain some context across
operators to be passed to the child which the walker will co-ordinate.
> NO PRECOMMIT TESTS (this is wip for the tez branch)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message