hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vineet Garg <vg...@hortonworks.com>
Subject Re: Review Request 62776: HIVE-16511 CBO looses inner casts on constants of complex type
Date Wed, 04 Oct 2017 22:06:14 GMT


> On Oct. 4, 2017, 9:18 p.m., Ashutosh Chauhan wrote:
> > ql/src/test/results/clientpositive/llap/vector_ptf_part_simple.q.out
> > Line 5616 (original), 5616 (patched)
> > <https://reviews.apache.org/r/62776/diff/1/?file=1846209#file1846209line5616>
> >
> >     I would have expected this expression to have been folded in compiler : CAST(
null AS TIMESTAMP) We have constant folding rules as both calcite as well as hive rules. 
> >     Any reason that's not happening here?
> 
> Vineet Garg wrote:
>     This patch introduces cast for nulls after generating logical plan (while converting
it to AST). That is why there is no constant folding. (In fact calcite constant folding is
folding such expressions does create null with appropriate type. But during converting logical
plan to ast we end up losing the type and create null with void type, probably because hive
doesn't associate type with nulls)
> 
> Ashutosh Chauhan wrote:
>     If we have proper type in Calcite, why do we lose type on translation back? Introducing
cast during AST generation feels hackish.

We lose type on translation back because hive doesn't associate any type with null literals.
e.g. select case when i=1 then i else null end from t will create GenericUDFWhen(GenericUDFOPEqual(Column[i],
Const int 1), Column[i], Const void null) expression for select expr with CBO disabled. As
you can notice null literal has void type instead of type corresponding to column[i]. So we
need to explicity introduce cast operator in AST to take care of this case. We were already
doing this before my patch, it's just the logic wasn't sophisticated enough to do it for expressions
embeded inside complex expressions like case, struct etc.


- Vineet


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62776/#review187130
-----------------------------------------------------------


On Oct. 4, 2017, 6:04 p.m., Vineet Garg wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62776/
> -----------------------------------------------------------
> 
> (Updated Oct. 4, 2017, 6:04 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan and Teddy Choi.
> 
> 
> Bugs: HIVE-16511
>     https://issues.apache.org/jira/browse/HIVE-16511
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> This patch fixes an issue with CBO where after generating logical plan null constants
will lose it's cast operator.
> 
> 
> Diffs
> -----
> 
>   itests/src/test/resources/testconfiguration.properties 038487f134 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java fcebb6fbce

>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ASTConverter.java
165f8c4317 
>   ql/src/test/results/clientpositive/llap/vector_coalesce.q.out d6865dd0ad 
>   ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_grouping.q.out
ddc1d7be7b 
>   ql/src/test/results/clientpositive/llap/vector_ptf_part_simple.q.out b63d1e4527 
>   ql/src/test/results/clientpositive/min_structvalue.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/perf/spark/query36.q.out c072728803 
>   ql/src/test/results/clientpositive/perf/spark/query70.q.out 716665bd4f 
>   ql/src/test/results/clientpositive/perf/spark/query86.q.out c5b25e8467 
>   ql/src/test/results/clientpositive/perf/tez/query36.q.out dee62b58f1 
>   ql/src/test/results/clientpositive/perf/tez/query70.q.out d700d60345 
>   ql/src/test/results/clientpositive/perf/tez/query86.q.out 6b59dd254f 
>   ql/src/test/results/clientpositive/vector_coalesce.q.out 445a246cb6 
> 
> 
> Diff: https://reviews.apache.org/r/62776/diff/1/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Vineet Garg
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message