hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Deepak Jaiswal <djais...@hortonworks.com>
Subject Re: Review Request 58973: HIVE-16578
Date Wed, 03 May 2017 22:39:36 GMT


> On May 3, 2017, 10:24 p.m., Jason Dere wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java
> > Lines 225 (patched)
> > <https://reviews.apache.org/r/58973/diff/1/?file=1707234#file1707234line225>
> >
> >     Can't totally tell the nesting here, but it seems like if getColumnName() returns
false this will not do semijoin reduction for this column, regardless of if there is a hint
or not .. is that intended?

Yes. From the old logic, in generateSemiJoinOperator(), we would return if we fail to obtain
the appropriate ExprNodeColumnDesc. Instead of returning there, now it never calls the function.
Basically, the logic has been pushed down.


> On May 3, 2017, 10:24 p.m., Jason Dere wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java
> > Lines 235 (patched)
> > <https://reviews.apache.org/r/58973/diff/1/?file=1707234#file1707234line237>
> >
> >     Remove your name from the log line.

Thanks! Forgot after debugging the code.


> On May 3, 2017, 10:24 p.m., Jason Dere wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java
> > Lines 306 (patched)
> > <https://reviews.apache.org/r/58973/diff/1/?file=1707234#file1707234line308>
> >
> >     Not sure if you can do this and truly say this is the column for this expression
- the expression could be a function, such as replace(col1, col2).
> >     
> >     How about ExprNodeDescUtils.getColumnExpr()?

Thanks for telling me about the method, will see if it works in this case.


> On May 3, 2017, 10:24 p.m., Jason Dere wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java
> > Lines 320 (patched)
> > <https://reviews.apache.org/r/58973/diff/1/?file=1707234#file1707234line322>
> >
> >     same here, about ExprNodeDescUtils.getColumnExpr()

ditto


- Deepak


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58973/#review173818
-----------------------------------------------------------


On May 3, 2017, 8:12 p.m., Deepak Jaiswal wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58973/
> -----------------------------------------------------------
> 
> (Updated May 3, 2017, 8:12 p.m.)
> 
> 
> Review request for hive and Jason Dere.
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> Semijoin Hints should use column name, if provided for partition key check.
> Involves some code refactoring.
> 
> 
> Diffs
> -----
> 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java
b8c01020b7 
> 
> 
> Diff: https://reviews.apache.org/r/58973/diff/1/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Deepak Jaiswal
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message