hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xuefu Zhang" <xzh...@cloudera.com>
Subject Re: Review Request 27283: Compile time skew join optimization doesn't work with auto map join
Date Wed, 29 Oct 2014 17:09:25 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27283/#review59001
-----------------------------------------------------------



ql/src/java/org/apache/hadoop/hive/ql/optimizer/SkewJoinOptimizer.java
<https://reviews.apache.org/r/27283/#comment100281>

    static method? Maybe we can create such clone() method in QBJoinTree class.



ql/src/java/org/apache/hadoop/hive/ql/optimizer/SkewJoinOptimizer.java
<https://reviews.apache.org/r/27283/#comment100282>

    Static method? Same to below



ql/src/java/org/apache/hadoop/hive/ql/optimizer/SkewJoinOptimizer.java
<https://reviews.apache.org/r/27283/#comment100279>

    The method seems to be recursive, but I'm not sure it's correct. We call the same method
for the parents and children of the current operator. When a parent or child operator of the
current operator is processed, wouldn't the process go back to the current operator again?
Recursion should be done in one direction and should have clear end condition.


- Xuefu Zhang


On Oct. 29, 2014, 2:59 a.m., Rui Li wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27283/
> -----------------------------------------------------------
> 
> (Updated Oct. 29, 2014, 2:59 a.m.)
> 
> 
> Review request for hive, Szehon Ho and Xuefu Zhang.
> 
> 
> Bugs: HIVE-8610
>     https://issues.apache.org/jira/browse/HIVE-8610
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> This patch adds QBJoinTree and colExprMap for the cloned join operator tree in SkewJoinOptimizer,
so that CommonJoinResolver can properly convert the cloned join to map join.
> The added tests are copied from skewjoinopt*.q, except that auto map join is enabled.
> 
> 
> Diffs
> -----
> 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/SkewJoinOptimizer.java e87c41b 
>   ql/src/test/queries/clientpositive/skewjoin_mapjoin1.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/skewjoin_mapjoin10.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/skewjoin_mapjoin11.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/skewjoin_mapjoin2.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/skewjoin_mapjoin3.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/skewjoin_mapjoin4.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/skewjoin_mapjoin5.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/skewjoin_mapjoin6.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/skewjoin_mapjoin7.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/skewjoin_mapjoin8.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/skewjoin_mapjoin9.q PRE-CREATION 
>   ql/src/test/results/clientpositive/skewjoin_mapjoin1.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/skewjoin_mapjoin10.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/skewjoin_mapjoin11.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/skewjoin_mapjoin2.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/skewjoin_mapjoin3.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/skewjoin_mapjoin4.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/skewjoin_mapjoin5.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/skewjoin_mapjoin6.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/skewjoin_mapjoin7.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/skewjoin_mapjoin8.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/skewjoin_mapjoin9.q.out PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/27283/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Rui Li
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message