Mailing-List: contact reviews-help@impala.incubator.apache.org; run by ezmlm
Precedence: bulk
Message-Id: <201710260607.v9Q67wFq027186@ip-10-146-233-104.ec2.internal>
Date: Thu, 26 Oct 2017 06:07:57 +0000
From: "Attila Jeges (Code Review)" <gerrit@cloudera.org>
To: impala-cr@cloudera.com, reviews@impala.incubator.apache.org
Subject: =?UTF-8?Q?=5BImpala-ASF-CR=5D_IMPALA-3548=3A_Prune_runtime_filters_based_on_query_options_in_the_FE=0A?=
In-Reply-To: <gerrit.1501677528000.Id0f0b200e02442edcad8df3979f652d66c6e52eb@gerrit.cloudera.org>
References: <gerrit.1501677528000.Id0f0b200e02442edcad8df3979f652d66c6e52eb@gerrit.cloudera.org>
Reply-To: attilaj@cloudera.com, impala-cr@cloudera.com, marcelk@gmail.com,
        reviews@impala.incubator.apache.org
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Disposition: inline
User-Agent: Gerrit/2.14.2
Content-Type: multipart/alternative; boundary="zqwiEl8LW9k="; charset=UTF-8
archived-at: Thu, 26 Oct 2017 06:08:04 -0000

--zqwiEl8LW9k=
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Attila Jeges has uploaded a new patch set (#13)=2E ( http://gerrit=2Ecloude=
ra=2Eorg:8080/7564 )

Change subject: IMPALA-3548: Prune runtime filters ba=
sed on query options in the FE
=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=
=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=
=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=
=2E=2E=2E=2E=2E=2E

IMPALA-3548: Prune runtime filters based on query optio=
ns in the FE

Currently, the FE generates a number of runtime filters and a=
ssigns
them to the single node plan without taking the value of
RUNTIME_FIL=
TER_MODE and DISABLE_ROW_RUNTIME_FILTERING query options
into account=2E

T=
he backend then removes filters from exec nodes, based on the
following rul=
es:
1=2E If DISABLE_ROW_RUNTIME_FILTERING is set, filters are removed from
=
   the exec nodes that are marked as targets not bound by partition
   colu=
mns=2E

2=2E If RUNTIME_FILTER_MODE is set to LOCAL, filters are removed fr=
om
   the exec nodes that are marked as remote targets=2E

This may cause s=
ome confusion to users because they may see runtime
filters in the output o=
f explain that are not applied when the query
is executed=2E

This change m=
oves the logic of runtime filter pruning to the planner
in the FE=2E The ru=
ntime filter assignment is done on the distributed
plan and the above const=
raints are enforced there directly=2E

Change-Id: Id0f0b200e02442edcad8df39=
79f652d66c6e52eb
---
M be/src/exec/partitioned-hash-join-builder=2Ecc
M be/=
src/exec/scan-node=2Ecc
M be/src/runtime/coordinator=2Ecc
M be/src/service/=
fe-support=2Ecc
M fe/src/main/java/org/apache/impala/planner/DistributedPla=
nner=2Ejava
M fe/src/main/java/org/apache/impala/planner/Planner=2Ejava
M f=
e/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator=2Ejava
M f=
e/src/main/java/org/apache/impala/service/FeSupport=2Ejava
M fe/src/test/ja=
va/org/apache/impala/planner/PlannerTest=2Ejava
M fe/src/test/java/org/apac=
he/impala/planner/PlannerTestBase=2Ejava
M fe/src/test/java/org/apache/impa=
la/testutil/TestFileParser=2Ejava
A testdata/workloads/functional-planner/q=
ueries/PlannerTest/runtime-filter-query-options=2Etest
12 files changed, 77=
5 insertions(+), 108 deletions(-)


  git pull ssh://gerrit=2Ecloudera=2Eor=
g:29418/Impala-ASF refs/changes/64/7564/13
-- 
To view, visit http://gerrit=
=2Ecloudera=2Eorg:8080/7564
To unsubscribe, visit http://gerrit=2Ecloudera=
=2Eorg:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerr=
it-MessageType: newpatchset
Gerrit-Change-Id: Id0f0b200e02442edcad8df3979f6=
52d66c6e52eb
Gerrit-Change-Number: 7564
Gerrit-PatchSet: 13
Gerrit-Owner: A=
ttila Jeges <attilaj@cloudera=2Ecom>
Gerrit-Reviewer: Alex Behm <alex=2Ebeh=
m@cloudera=2Ecom>
Gerrit-Reviewer: Attila Jeges <attilaj@cloudera=2Ecom>
Ge=
rrit-Reviewer: Dimitris Tsirogiannis <dtsirogiannis@cloudera=2Ecom>
Gerrit-=
Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Matthew Jacobs <mjacobs@ap=
ache=2Eorg>
Gerrit-Reviewer: Michael Ho <kwho@cloudera=2Ecom>
Gerrit-Review=
er: Philip Zeyliger <philip@cloudera=2Ecom>
Gerrit-Reviewer: Thomas Tauber-=
Marshall <tmarshall@cloudera=2Ecom>

--zqwiEl8LW9k=--