impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Behm (Code Review)" <>
Subject [Impala-ASF-CR] IMPALA-4309: Introduce Expr rewrite phase and supporting classes.
Date Mon, 24 Oct 2016 21:14:09 GMT
Alex Behm has posted comments on this change.

Change subject: IMPALA-4309: Introduce Expr rewrite phase and supporting classes.

Patch Set 5:

File fe/src/main/java/org/apache/impala/analysis/

Line 355
> what was going on with this?
This was stale code from the first version of subquery rewriting which was later cleaned up.
It's not needed.

Line 379:         analysisResult_.stmt_.rewriteExprs(rewriter_);
> for the between transformation it's not necessary, but in general i'd expec
I was thinking that we'd cross that bridge when our set of rewrite rules becomes complex enough
to warrant that, but easy enough to do now. Done.
File fe/src/main/java/org/apache/impala/analysis/

Line 56:           "supported in a between predicate: " + toSqlImpl());
> capitalize between
File fe/src/main/java/org/apache/impala/analysis/

Line 59:   protected Expr havingClause_;  // original having clause
> why move it?

Line 874:       // Also rewrite exprs in the statements of subqueries.
> the problem you have is that the parent/child expr tree doesn't give you ac
Expr.getContainedExprs() does not help because we also need to set the rewritten exprs in
the appropriate clauses of the contained QueryStmt. Sure, we can get the exprs and rewrite
them, but then we don't know where to set them. We could make everything Reference<Expr>.

Let me know if you want to go down the "everything is a ParseNode path" and I'll abandon this
patch. That will be much more involved change and I personally don't think it's worth it.
That change will save us these < 50 LOC to traverse the stmts and I don't see us rewriting
non-Expr ParseNodes with that new infra (we'll likely keep our existing StmtRewriter).
File fe/src/main/java/org/apache/impala/analysis/

Line 69
> huh?
Moved to caller to avoid several reset() and reanalyze() passes.
File fe/src/main/java/org/apache/impala/rewrite/

Line 38:     numChanges_ = 0;
> should rules really have state?
1. Sure, the driver can keep track of changes instead, but it seems like we'll need excessive
object creation:

for (Rule rule: rules) {
  Expr clone = expr.clone();
  clone = rule.apply(clone);
  changed = clone.equals(expr);

I made those changes.

2. I think there are some rewrite rules that are way easier to express if the rule itself
can traverse the entire Expr tree. For example, removing redundant conjuncts or disjuncts
seems a little cumbersome in your proposed approach. Seems like we'd need to have a normalization
pass first to get the redundant exprs next to each other so a single rule application can
detect the redundancy.

Line 46:     if (expr instanceof BetweenPredicate) {
> if (!(expr instanceof ...)) return expr;
File fe/src/main/java/org/apache/impala/rewrite/

Line 40:   public abstract Expr rewrite(Expr expr, Analyzer analyzer) throws AnalysisException;
> call it apply then?
File fe/src/main/java/org/apache/impala/rewrite/

Line 31: public class ExprRewriter extends ExprRewriteRule {
> what's the point of making it a subclass of ExprRewriteRule?
1. No need to make subclass ExprRewriteRule. Done.

2. I don't think we want to hardcode the rules in here because we will want to do different
sets of rewrites in different phases of planning. For example, the BETWEEN rewrite is a logical
rewrite that we can do on the parse "tree". But constant folding will need to be done on the
fully substituted Exprs, i.e., during planning. Of course, we can just brute-force it and
always apply everything but I think it's easier to understand if we have a more nuanced application
of rules.

3. I made the other changed you requested here:
* Keep applying rules until there no more changes.
* Have the driver traverse the expr tree and detect changes.
File fe/src/test/java/org/apache/impala/common/

Line 186:     // Do not analyze the stmt to avoid applying rewrites.
> why is that bad?
Added comment.

To view, visit
To unsubscribe, visit

Gerrit-MessageType: comment
Gerrit-Change-Id: I2279dc984bcf7742db4fa3b1aa67283ecbb05e6e
Gerrit-PatchSet: 5
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Alex Behm <>
Gerrit-Reviewer: Alex Behm <>
Gerrit-Reviewer: Bharath Vissapragada <>
Gerrit-Reviewer: Dimitris Tsirogiannis <>
Gerrit-Reviewer: Marcel Kornacker <>
Gerrit-Reviewer: Tim Armstrong <>
Gerrit-HasComments: Yes

View raw message