hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Dere" <jd...@hortonworks.com>
Subject Re: Review Request 36486: HIVE-11262 Skip MapJoin processing if the join hash table is empty
Date Thu, 16 Jul 2015 00:45:25 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36486/
-----------------------------------------------------------

(Updated July 16, 2015, 12:45 a.m.)


Review request for hive, Matt McCline, Vikram Dixit Kumaraswamy, and Wei Zheng.


Changes
-------

Updating patch regarding size of HybridHashTableContainer, based on feedback from Wei Zheng
- the on-disk hash partition size is based on both the on-disk hash table as well as the side
table. Added a new field to keep track of the on-disk hash table size.
Also made a couple of small fixes to HybridHashTableContainer:
In MapJoin.reloadHashTable(), totalInMemRowCount was double counting the size of the side
table, since the restoreHashMap had already added the side table values.
hashMapOnDisk was not being reset to false when the on-disk hash table was being cleaned up.


Bugs: HIVE-11262
    https://issues.apache.org/jira/browse/HIVE-11262


Repository: hive-git


Description
-------

- Added size() method to HashTableContainer interface/implementations.
- After loading hashTable, check if size == 0 and if join is all inner joins. If so, set done
on the MapJoinOperator.


Diffs (updated)
-----

  ql/src/java/org/apache/hadoop/hive/ql/exec/MapJoinOperator.java 15cafdd 
  ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/HybridHashTableContainer.java e338a31

  ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinBytesTableContainer.java 83a1521

  ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinTableContainer.java 9d8cbcb

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastHashTable.java
fbe6b4c 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastTableContainer.java
4b1d6f6 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/hashtable/VectorMapJoinHashTable.java
7e219ec 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/optimized/VectorMapJoinOptimizedHashTable.java
a2d4e4c 

Diff: https://reviews.apache.org/r/36486/diff/


Testing
-------


Thanks,

Jason Dere


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message