hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vikram Dixit Kumaraswamy" <>
Subject Review Request 13787: HIVE-5095: Hive needs new operator walker for parallelization/optimization for tez
Date Fri, 23 Aug 2013 23:45:23 GMT

This is an automatically generated e-mail. To reply, visit:

Review request for hive and Gunther Hagleitner.

Bugs: HIVE-5095

Repository: hive-git


For tez to compute the number of reducers, we should be walking the operator tree in a topological
fashion so that the reducers down the tree get the estimate from all parents. However, the
current walkers in hive only walk the operator tree in a depth-first fashion. We need to add
a new walker for the topological walk. Also, since information about the parent operators
needs to be propagated on a per parent basis, we need to retain some context across operators
to be passed to the child which the walker will co-ordinate.


  common/src/java/org/apache/hadoop/hive/conf/ 7408a5a 
  ql/src/java/org/apache/hadoop/hive/ql/exec/ ca48f5e 
  ql/src/java/org/apache/hadoop/hive/ql/exec/ 6a538e8 
  ql/src/java/org/apache/hadoop/hive/ql/exec/ 6ee13ec 
  ql/src/java/org/apache/hadoop/hive/ql/exec/ f3c34d1 
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ 49a0ee3 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ 7433ddc 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/ PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/parse/ PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/parse/ 827637a 
  ql/src/java/org/apache/hadoop/hive/ql/parse/ ff8b17b 
  ql/src/java/org/apache/hadoop/hive/ql/parse/ c1c1da5 
  ql/src/java/org/apache/hadoop/hive/ql/parse/ 248eb03 
  ql/src/java/org/apache/hadoop/hive/ql/parse/ 5abedfe 
  ql/src/java/org/apache/hadoop/hive/ql/plan/ 5fd8d828 




Vikram Dixit Kumaraswamy

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message