Mailing-List: contact reviews-help@impala.incubator.apache.org; run by ezmlm
Precedence: bulk
Message-Id: <201703312325.v2VNPQpH010179@ip-10-146-233-104.ec2.internal>
Date: Fri, 31 Mar 2017 23:25:26 +0000
From: "Zach Amsden (Code Review)" <gerrit@cloudera.org>
To: impala-cr@cloudera.com, reviews@impala.incubator.apache.org
CC: Marcel Kornacker <marcel@cloudera.com>, Alex Behm <alex.behm@cloudera.com>,
        Dan Hecht <dhecht@cloudera.com>
Reply-To: zamsden@cloudera.com
Subject: =?UTF-8?Q?=5BImpala-ASF-CR=5D_IMPALA-5003=3A_Constant_propagation_in_scan_nodes_and_inline_views=0A?=
In-Reply-To: <gerrit.1489511537000.I79750a8edb945effee2a519fa3b8192b77042cb4@gerrit.cloudera.org>
References: <gerrit.1489511537000.I79750a8edb945effee2a519fa3b8192b77042cb4@gerrit.cloudera.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Content-Disposition: inline
User-Agent: Gerrit/2.12.7
archived-at: Fri, 31 Mar 2017 23:25:31 -0000

Zach Amsden has uploaded a new patch set (#11).

Change subject: IMPALA-5003: Constant propagation in scan nodes and inline views
......................................................................

IMPALA-5003: Constant propagation in scan nodes and inline views

When conjuncts are pushed into table refs and inline views, they can
be considered for constant progagation within that node.  In certain
cases, we might end up with a FALSE conditional and now we can
convert ScanNodes to EmptySet nodes when that occurs.

I also added an inequality collation phase which is now partially
tested and will combine conjuncts such as a < k1, a < k2  into
a < min(k1, k2), as well as detect equivalence from a >= k, a <= k,
and determine conflicting bounds requirements to be false.

This could be expanded to do analysis against other slotrefs in the
future, but this should probably be saved for another diff.

Testing: Expanded the test cases for the planner to achieve constant
propagation.  Added Kudu, datasource, Hdfs and HBase tests to validate
we can create EmptySetNodes.  Some manual testing for inequality
conjuncts but nothing formal yet.

Query: explain select * from functional_hbase.widetable_250_cols a
where a.int_col1 > 1 and a.int_col1 <= 20 and a.int_col1 < 50 and
a.int_col1 > 2
+-------------------------------------------------------------------
| Explain String
+-------------------------------------------------------------------
| Estimated Per-Host Requirements: Memory=1.00GB VCores=1
| PLAN-ROOT SINK
| |
| 01:EXCHANGE [UNPARTITIONED]
| |
| 00:SCAN HBASE [functional_hbase.widetable_250_cols a]
|    predicates: a.int_col1 <= 20, a.int_col1 > 2
+-------------------------------------------------------------------
Fetched 10 row(s) in 0.08s

Change-Id: I79750a8edb945effee2a519fa3b8192b77042cb4
---
M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/analysis/Expr.java
M fe/src/main/java/org/apache/impala/analysis/SelectList.java
M fe/src/main/java/org/apache/impala/planner/DataSourceScanNode.java
M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java
M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java
M fe/src/main/java/org/apache/impala/planner/ValueRange.java
M fe/src/main/java/org/apache/impala/rewrite/ExprRewriter.java
M fe/src/main/java/org/apache/impala/rewrite/NormalizeBinaryPredicatesRule.java
M fe/src/test/java/org/apache/impala/planner/PlannerTest.java
M testdata/workloads/functional-planner/queries/PlannerTest/analytic-fns.test
M testdata/workloads/functional-planner/queries/PlannerTest/conjunct-ordering.test
A testdata/workloads/functional-planner/queries/PlannerTest/constant-propagation.test
M testdata/workloads/functional-planner/queries/PlannerTest/data-source-tables.test
M testdata/workloads/functional-planner/queries/PlannerTest/hdfs.test
M testdata/workloads/functional-planner/queries/PlannerTest/joins.test
M testdata/workloads/functional-planner/queries/PlannerTest/kudu.test
M testdata/workloads/functional-planner/queries/PlannerTest/subquery-rewrite.test
19 files changed, 644 insertions(+), 117 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/89/6389/11
-- 
To view, visit http://gerrit.cloudera.org:8080/6389
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I79750a8edb945effee2a519fa3b8192b77042cb4
Gerrit-PatchSet: 11
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Zach Amsden <zamsden@cloudera.com>
Gerrit-Reviewer: Alex Behm <alex.behm@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhecht@cloudera.com>
Gerrit-Reviewer: Marcel Kornacker <marcel@cloudera.com>
Gerrit-Reviewer: Zach Amsden <zamsden@cloudera.com>