hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suhas Satish" <suhas.sat...@gmail.com>
Subject Re: Review Request 27247: convert joinOp to MapJoinOp and generate MapWorks only
Date Wed, 29 Oct 2014 06:24:07 GMT


> On Oct. 27, 2014, 11:03 p.m., Szehon Ho wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/parse/spark/SparkCompiler.java, line 151
> > <https://reviews.apache.org/r/27247/diff/1/?file=734616#file734616line151>
> >
> >     I don't see this class attached

This was already included as a part of optimizer/spark/SparkReduceSinkMapJoinProc.java

Enabling the rule automatically started using it.


> On Oct. 27, 2014, 11:03 p.m., Szehon Ho wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/parse/spark/SparkCompiler.java, line 118
> > <https://reviews.apache.org/r/27247/diff/1/?file=734616#file734616line118>
> >
> >     We need to put the check of auto.convert.join before we enable this optimization.
 It would make it easier this way.

I think its better to make auto.convert.join true by default and hence, map-join true by default.
Have you seen a use case which would specifically want to disable map joins?


- Suhas


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27247/#review58712
-----------------------------------------------------------


On Oct. 27, 2014, 9:53 p.m., Suhas Satish wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27247/
> -----------------------------------------------------------
> 
> (Updated Oct. 27, 2014, 9:53 p.m.)
> 
> 
> Review request for hive, Rui Li, Szehon Ho, and Xuefu Zhang.
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> This addresses jira HIVE-8616
> 
> 
> Diffs
> -----
> 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/MapJoinProcessor.java 46dcfaf 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/spark/SparkConvertJoinMapJoin.java
PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/spark/GenSparkProcContext.java ed88c60

>   ql/src/java/org/apache/hadoop/hive/ql/parse/spark/GenSparkUtils.java 8e28887 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/spark/GenSparkWork.java 4f5feca 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/spark/SparkCompiler.java 1c663c4 
> 
> Diff: https://reviews.apache.org/r/27247/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Suhas Satish
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message