hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jesus Camacho Rodriguez (JIRA)" <>
Subject [jira] [Commented] (HIVE-13068) Disable Hive ConstantPropagate optimizer when CBO has optimized the plan II
Date Fri, 22 Apr 2016 19:27:12 GMT


Jesus Camacho Rodriguez commented on HIVE-13068:

I agree with you.

However, I think it is difficult because of the current limitations that I outlined above.

- For instance, is there a way to specify that a given constant should be of a certain type
in HQL without using cast/udfto? I think there is not, that is why I came with this solution.
I could try to add the logic to fold the expression to the SemanticAnalyzer, thus not traversing
the tree. But I think it will not make such a big difference performance wise and we will
end up adding more logic to SemanticAnalyzer, which is already huge. Further, if return path
is in place at some point, it will be more difficult to detect/remove that unnecessary logic

- The problem with {{power}} expression is that it was returning NaN when it was folded, and
I think we cannot convert that currently through AST? It was a corner case and I did not explore
it much further as the patch was so full of small pieces. But I will explore it further.

- The filter removal for {{true}} predicates comes practically for free if we need to execute
SimpleConstantReduction, but I agree I could try to rewrite that logic in SemanticAnalyzer.

The fact that we are not carrying constants all over the operator tree and checking all the
expressions to replace them, should make a performance difference already. I agree with you
that we need to take it a step further, but without further integration work, I did not figure
out a better solution for some of these cases (specially the CAST, that is quite common because
of Calcite tight type system).

> Disable Hive ConstantPropagate optimizer when CBO has optimized the plan II
> ---------------------------------------------------------------------------
>                 Key: HIVE-13068
>                 URL:
>             Project: Hive
>          Issue Type: Sub-task
>          Components: CBO, Logical Optimizer
>    Affects Versions: 2.1.0
>            Reporter: Jesus Camacho Rodriguez
>            Assignee: Jesus Camacho Rodriguez
>         Attachments: HIVE-13068.01.patch, HIVE-13068.01.patch, HIVE-13068.patch
> After HIVE-12543 went in, we need follow-up work to disable the last call to ConstantPropagate
in Hive. This probably implies work on extending the constant folding logic in Calcite.

This message was sent by Atlassian JIRA

View raw message