hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John Pullokkaran" <jpullokka...@hortonworks.com>
Subject Re: Review Request 29763: HIVE-9292
Date Tue, 13 Jan 2015 23:38:37 GMT


> On Jan. 13, 2015, 6:59 p.m., John Pullokkaran wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java, line 96
> > <https://reviews.apache.org/r/29763/diff/3/?file=818832#file818832line96>
> >
> >     It seems like groupOpToInputTables is used only by RewriteQueryUsingAggregateIndexCtx.
Couldn't we get rid of groupToInputTables from parse context by changing "RewriteQueryUsingAggregateIndexCtx"?
> 
> Jesús Camacho Rodríguez wrote:
>     The method "replaceGroupByOperatorProcess" 1) creates a query in String format, 2)
uses "RewriteParseContextGenerator" to create an operator tree from it, and 3) then takes
the aggregator from the new operator tree and replaces the old aggregation with it. It does
not seem the most elegant way to create the replacement aggregator.
>     Do you think that I should rely on any other structure that is already in ParseContext
to retrieve the GroupByOperator so the process remains unchanged, or rather you prefer that
I rewrite it completely?

The optimization is trying to rewrite GB-TS as GB on IndexTable so as to reduce the scan &
aggregation cost. That may be the reason why it generates query string first. You may not
want to change this instead just change how it gets hold of GB op (from tree as opposed to
parsecontext).


- John


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29763/#review67910
-----------------------------------------------------------


On Jan. 13, 2015, 9:17 a.m., Jesús Camacho Rodríguez wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/29763/
> -----------------------------------------------------------
> 
> (Updated Jan. 13, 2015, 9:17 a.m.)
> 
> 
> Review request for hive and John Pullokkaran.
> 
> 
> Bugs: HIVE-9292
>     https://issues.apache.org/jira/browse/HIVE-9292
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> CBO (Calcite Return Path): Inline GroupBy, Properties
> 
> 
> Diffs
> -----
> 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/index/RewriteQueryUsingAggregateIndexCtx.java
fe686d96b642572059ef13129951d01fce4fedce 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java 8215c26f48f1611d3c64f6df5bcfac02069e3a67

>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java cea86dfbf67b85cba24fb0e7ebf270abbe9c31f9

>   ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java 23fbbe11198ac5893a84bdf94f9c843c4ee2ccb4

>   ql/src/java/org/apache/hadoop/hive/ql/plan/GroupByDesc.java 7a0b0da5de5c1c7dc8dd099167e9d06e6b27eea2

> 
> Diff: https://reviews.apache.org/r/29763/diff/
> 
> 
> Testing
> -------
> 
> Existing tests.
> 
> 
> Thanks,
> 
> Jesús Camacho Rodríguez
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message