hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashutosh Chauhan" <hashut...@apache.org>
Subject Re: Review Request 14953: Pushdown join conditions
Date Thu, 31 Oct 2013 00:34:00 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14953/#review27840
-----------------------------------------------------------



ql/src/test/results/clientpositive/join_cond_pushdown_1.q.out
<https://reviews.apache.org/r/14953/#comment54177>

    Ya, this will amount to taking into account filter condition while actually performing
join. Which implies enhancing physical Join Operator to do it. In absence of that, what we
have currently is best we can do.



ql/src/test/results/clientpositive/join_cond_pushdown_2.q.out
<https://reviews.apache.org/r/14953/#comment54176>

    Can this be done as follows:
    First MR job:
     Mapper reads P1,P2,P3 and partitions by partname
    In reducer 3-way join of P1,P2,P3 on partname, followed by post-join filter of p1.p_partkey=p2.p_partkey
    
    Second MR job:
    Mapper reads output of previous job and P4 and partition both sides by p_partkey and than
join them on p_partkey in reducer. 
    Seems like this will generate correct results.
    


- Ashutosh Chauhan


On Oct. 29, 2013, 9:19 p.m., Harish Butani wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14953/
> -----------------------------------------------------------
> 
> (Updated Oct. 29, 2013, 9:19 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan and Vikram Dixit Kumaraswamy.
> 
> 
> Bugs: hive-5556
>     https://issues.apache.org/jira/browse/hive-5556
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> Step 1 to support Alternate Join Syntax: HIVE-5555
> 
> This patch also contains fixes to merging of QBJoinTrees
> 
> 
> Diffs
> -----
> 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/QBJoinTree.java 9c8cac1 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java cf0c895 
>   ql/src/test/org/apache/hadoop/hive/ql/parse/TestQBJoinTreeApplyPredicate.java PRE-CREATION

>   ql/src/test/queries/clientpositive/join_cond_pushdown_1.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/join_cond_pushdown_2.q PRE-CREATION 
>   ql/src/test/results/clientpositive/auto_sortmerge_join_12.q.out 865627b 
>   ql/src/test/results/clientpositive/join_cond_pushdown_1.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/join_cond_pushdown_2.q.out PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/14953/diff/
> 
> 
> Testing
> -------
> 
> ran all join .q files
> added join_cond_pushdown_1.q, join_cond_pushdown_2.q .q tests
> added TestQBJoinTreeApplyPredicate unit test to test pushdown functionality
> 
> 
> Thanks,
> 
> Harish Butani
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message