hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin" <ser...@hortonworks.com>
Subject Re: Review Request 24427: HIVE-7616 pre-size mapjoin hashtable based on statistics
Date Thu, 07 Aug 2014 22:11:37 GMT


> On Aug. 7, 2014, 12:04 a.m., Gunther Hagleitner wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/ReduceSinkMapJoinProc.java, line
136
> > <https://reviews.apache.org/r/24427/diff/2/?file=654266#file654266line136>
> >
> >     curlies per coding standard

added; next time I review your patch, I'll enforce C variable declarations (all variables
in the beginning of the block) which are also part of the same Sun standard Hive wiki page
point to :P


> On Aug. 7, 2014, 12:04 a.m., Gunther Hagleitner wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/ReduceSinkMapJoinProc.java, line
146
> > <https://reviews.apache.org/r/24427/diff/2/?file=654266#file654266line146>
> >
> >     i think this number needs to be adjusted for bucketed map join. otherwise you'll
over allocate in that case, but we take the bucketing into consideration when we do size estimation
for the overall operator...

can you elaborate on "but we take the bucketing into consideration when we do size estimation
for the overall operator..."? 


- Sergey


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24427/#review49830
-----------------------------------------------------------


On Aug. 6, 2014, 10 p.m., Sergey Shelukhin wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24427/
> -----------------------------------------------------------
> 
> (Updated Aug. 6, 2014, 10 p.m.)
> 
> 
> Review request for hive, Gunther Hagleitner, Mostafa Mokhtar, and Prasanth_J.
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> See jira
> 
> 
> Diffs
> -----
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 8490558 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/HashTableSinkOperator.java cf64aa0 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/BytesBytesMultiHashMap.java
cdb5dc5 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/HashMapWrapper.java 5b3b770

>   ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinBytesTableContainer.java
629457c 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/tez/HashTableLoader.java 6d292d0 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/ReduceSinkMapJoinProc.java 29d895a

>   ql/src/java/org/apache/hadoop/hive/ql/plan/MapJoinDesc.java 44cb9c0 
> 
> Diff: https://reviews.apache.org/r/24427/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Sergey Shelukhin
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message