Mailing-List: contact reviews-help@impala.incubator.apache.org; run by ezmlm
Precedence: bulk
Message-Id: <201611071916.uA7JG8jh011482@ip-10-146-233-104.ec2.internal>
Date: Mon, 7 Nov 2016 19:16:07 +0000
From: "Alex Behm (Code Review)" <gerrit@cloudera.org>
To: impala-cr@cloudera.com, reviews@impala.incubator.apache.org
Reply-To: alex.behm@cloudera.com
Subject: =?UTF-8?Q?=5BImpala-ASF-CR=5D_IMPALA-3167=3A_Fix_assignment_of_WHERE-clause_predicate_through_grouping_agg_+_outer_join=2E=0A?=
In-Reply-To: <gerrit.1478299975000.I774d13a13ad1e8fe82512df98dc29983bdd232eb@gerrit.cloudera.org>
References: <gerrit.1478299975000.I774d13a13ad1e8fe82512df98dc29983bdd232eb@gerrit.cloudera.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Content-Disposition: inline
User-Agent: Gerrit/2.12.2
archived-at: Mon, 07 Nov 2016 19:16:44 -0000

Alex Behm has uploaded a new patch set (#2).

Change subject: IMPALA-3167: Fix assignment of WHERE-clause predicate through grouping agg + outer join.
......................................................................

IMPALA-3167: Fix assignment of WHERE-clause predicate through grouping agg + outer join.

Background: We generally allow the assignment of predicates below the nullable side
of a left/right outer join, explained as follows using an example:

SELECT * FROM t1 LEFT OUTER JOIN t2 ON t1.id = t2.id
WHERE t2.int_col < 10

The scan of 't2' will pick up 't2.int_col < 10' via Analyzer.getBoundPredicates()
and recognizes that the predicate must also be evaluated by a join later, so the
predicate is not marked as assigned. The join then picks up the unassigned
predicate via Analyzer.getUnassignedConjuncts().

The bug was that our logic for detecting whether a bound predicate must also be
evaluated at a join node was flawed because it only considered whether the tuples
of the source or destination predicate were outer joined (plus other conditions).
The underlying assumption is that either the source or destination tuple are bound
by a tuple produced by a TableRef, but in the buggy query the source predicate
is bound by an aggregation tuple, so we incorrectly marked the bound predicate
as assigned in Analyzer.getBoundPredicates().

The fix is to conservatively not mark bound predicates as assigned if there exists
equivalent tuples that are outer joined. This conservative fix leads to some
duplicate assignments of predicates. Those are simply deduped now.

Change-Id: I774d13a13ad1e8fe82512df98dc29983bdd232eb
---
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/analysis/Expr.java
M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java
M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java
M testdata/workloads/functional-planner/queries/PlannerTest/outer-joins.test
6 files changed, 48 insertions(+), 27 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/60/4960/2
-- 
To view, visit http://gerrit.cloudera.org:8080/4960
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I774d13a13ad1e8fe82512df98dc29983bdd232eb
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Alex Behm <alex.behm@cloudera.com>
Gerrit-Reviewer: Alex Behm <alex.behm@cloudera.com>
Gerrit-Reviewer: Anonymous Coward #27