hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vikram Dixit Kumaraswamy" <vikram.di...@gmail.com>
Subject Re: Review Request 12827: HIVE-4611 - SMB joins fail based on bigtable selection policy.
Date Mon, 05 Aug 2013 17:57:13 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12827/
-----------------------------------------------------------

(Updated Aug. 5, 2013, 5:57 p.m.)


Review request for hive, Ashutosh Chauhan, Brock Noland, and Gunther Hagleitner.


Changes
-------

Addressed Gunther's comments.


Bugs: HIVE-4611
    https://issues.apache.org/jira/browse/HIVE-4611


Repository: hive-git


Description
-------

SMB joins fail based on bigtable selection policy. The default setting for hive.auto.convert.sortmerge.join.bigtable.selection.policy
will choose the big table as the one with largest average partition size. However, this can
result in a query failing because this policy conflicts with the big table candidates chosen
for outer joins. This policy should just be a tie breaker and not have the ultimate say in
the choice of tables.


Diffs (updated)
-----

  ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java 12e9334 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/AbstractSMBJoinProc.java cc9de54 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/AvgPartitionSizeBasedBigTableSelectorForAutoSMJ.java
5320143 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/BigTableSelectorForAutoSMJ.java db5ff0f

  ql/src/java/org/apache/hadoop/hive/ql/optimizer/LeftmostBigTableSelectorForAutoSMJ.java
db3c9e7 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/MapJoinProcessor.java cd1b4ad 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/TableSizeBasedBigTableSelectorForAutoSMJ.java
b882f87 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/correlation/CorrelationOptimizer.java 3071713

  ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/CommonJoinTaskDispatcher.java e214807

  ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SortMergeJoinTaskDispatcher.java
da5115b 
  ql/src/test/queries/clientnegative/auto_sortmerge_join_1.q c858254 
  ql/src/test/queries/clientpositive/auto_sortmerge_join_15.q PRE-CREATION 
  ql/src/test/results/clientnegative/auto_sortmerge_join_1.q.out 0eddb69 
  ql/src/test/results/clientnegative/smb_bucketmapjoin.q.out 7a5b8c1 
  ql/src/test/results/clientpositive/auto_sortmerge_join_15.q.out PRE-CREATION 

Diff: https://reviews.apache.org/r/12827/diff/


Testing
-------

All tests pass on hadoop 1.


Thanks,

Vikram Dixit Kumaraswamy


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message