Mailing-List: contact reviews-help@impala.incubator.apache.org; run by ezmlm
Precedence: bulk
Message-Id: <201611030522.uA35MVsO020021@ip-10-146-233-104.ec2.internal>
Date: Thu, 3 Nov 2016 05:22:31 +0000
From: "Alex Behm (Code Review)" <gerrit@cloudera.org>
To: impala-cr@cloudera.com, reviews@impala.incubator.apache.org
CC: Bharath Vissapragada <bharathv@cloudera.com>,
        Marcel Kornacker <marcel@cloudera.com>,
        Tim Armstrong <tarmstrong@cloudera.com>
Reply-To: alex.behm@cloudera.com
Subject: =?UTF-8?Q?=5BImpala-ASF-CR=5D_IMPALA-1286=3A_Extract_common_conjuncts_from_disjunctions=2E=0A?=
In-Reply-To: <gerrit.1477666346000.I3cf9b950afaa3fd753d1b09ba5e540b5258940ad@gerrit.cloudera.org>
References: <gerrit.1477666346000.I3cf9b950afaa3fd753d1b09ba5e540b5258940ad@gerrit.cloudera.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Content-Disposition: inline
User-Agent: Gerrit/2.12.2
archived-at: Thu, 03 Nov 2016 05:22:38 -0000

Alex Behm has posted comments on this change.

Change subject: IMPALA-1286: Extract common conjuncts from disjunctions.
......................................................................


Patch Set 2:

(7 comments)

http://gerrit.cloudera.org:8080/#/c/4877/2/fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java
File fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java:

Line 70:         BetweenToCompoundRule.INSTANCE, ExtractCommonConjunctRule.INSTANCE);
> for added effectiveness you want the between rule to be applied before the 
Done


http://gerrit.cloudera.org:8080/#/c/4877/2/fe/src/main/java/org/apache/impala/analysis/Subquery.java
File fe/src/main/java/org/apache/impala/analysis/Subquery.java:

Line 155:     return stmt_.toSql().equals(((Subquery)o).stmt_.toSql());
> hm, that seems a bit fragile, won't insignificant syntactic differences (su
Agree, somewhat fragile. But correct.

This change exposed an existing bug involving the lack of a proper Subquery.equals(). I filed a separate JIRA and CR for that bug. Added the TODO to that patch which should go in before this one.

Bugfix CR:
http://gerrit.cloudera.org:8080/4923


http://gerrit.cloudera.org:8080/#/c/4877/1/fe/src/main/java/org/apache/impala/rewrite/ExtractCommonConjunctRule.java
File fe/src/main/java/org/apache/impala/rewrite/ExtractCommonConjunctRule.java:

Line 31:  * Extracts common conjuncts from a single disjunctive CompoundPredicate.
> I agree we don't need to document the framework here, but I think it's non-
Used your wording verbatim.


http://gerrit.cloudera.org:8080/#/c/4877/2/fe/src/main/java/org/apache/impala/rewrite/ExtractCommonConjunctRule.java
File fe/src/main/java/org/apache/impala/rewrite/ExtractCommonConjunctRule.java:

Line 51:     List<Expr> child0Conjuncts = expr.getChild(0).getConjuncts();
> checkstate that the child conjuncts aren't empty?
Added Preconditions check.

Like Tim mentioned this sounds like a "simplification" normalization rule that would work together with constant folding.

Arguably, your example does not fit this rule because there are no common conjuncts.


Line 51:     List<Expr> child0Conjuncts = expr.getChild(0).getConjuncts();
> true or x => true seems like a normalisation rule we should add in a follow
Good point.


Line 62:         conjunct.setPrintSqlInParens(false);
> add comment: why
Done


http://gerrit.cloudera.org:8080/#/c/4877/1/fe/src/test/java/org/apache/impala/analysis/ExprRewriteRulesTest.java
File fe/src/test/java/org/apache/impala/analysis/ExprRewriteRulesTest.java:

Line 146:         "(!(int_col=5 or tinyint_col > 9) and double_col = 7) or " +
> I think we should just be mindful that this change without normalisation ad
Good point about the perf cliff. I think it's doable to squeeze in a few simple normalization rules, e.g. for BinaryPredicates.

Agree that constant folding will need normalization as well for maximal effectiveness, e.g., 1 + a + 10

I was actually thinking of doing constant folding next (without handling that case above), unless you want to take it on instead.


-- 
To view, visit http://gerrit.cloudera.org:8080/4877
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I3cf9b950afaa3fd753d1b09ba5e540b5258940ad
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Alex Behm <alex.behm@cloudera.com>
Gerrit-Reviewer: Alex Behm <alex.behm@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bharathv@cloudera.com>
Gerrit-Reviewer: Marcel Kornacker <marcel@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>
Gerrit-HasComments: Yes