hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John Pullokkaran" <jpullokka...@hortonworks.com>
Subject Re: Review Request 38211: HIVE-11684
Date Thu, 08 Oct 2015 20:09:59 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38211/#review101952
-----------------------------------------------------------



ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSortJoinReduceRule.java
(line 4)
<https://reviews.apache.org/r/38211/#comment159463>

    As we discussed:
    we should seperate Limit push down from constraining condition. 
    
    I.e first push limit as low as possible then pull it up in the subsequent phase if it
doesn't meet reduction criteria.
    
    Also the reduction criteria probably needs to specify the minimun no of tuples to reduce
for limit to be cost effective (this should go away once we have execution cost model).


- John Pullokkaran


On Sept. 28, 2015, 11:40 a.m., Jesús Camacho Rodríguez wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38211/
> -----------------------------------------------------------
> 
> (Updated Sept. 28, 2015, 11:40 a.m.)
> 
> 
> Review request for hive and John Pullokkaran.
> 
> 
> Bugs: HIVE-11684
>     https://issues.apache.org/jira/browse/HIVE-11684
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> Implement limit pushdown through outer join in CBO
> 
> 
> Diffs
> -----
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java dffdb5c14114d9c2d1e4c44a36f6b60966dcfc86

>   itests/src/test/resources/testconfiguration.properties 4f7b25f2563683933b9d677626f0447705927966

>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSortJoinReduceRule.java
PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSortProjectTransposeRule.java
PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 9c731b8c5e58570570812b525b8bf0edb0f4c0a9

>   ql/src/test/queries/clientpositive/limit_join_transpose.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/tez_dynpart_hashjoin_3.q PRE-CREATION 
>   ql/src/test/results/clientpositive/limit_join_transpose.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/tez/tez_dynpart_hashjoin_3.q.out PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/38211/diff/
> 
> 
> Testing
> -------
> 
> Two new tests added: limit_join_transpose.q and tez_dynpart_hashjoin_3.q (only Tez)
> 
> 
> Thanks,
> 
> Jesús Camacho Rodríguez
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message