Mailing-List: contact reviews-help@impala.apache.org; run by ezmlm
Precedence: bulk
Message-Id: <201712120010.vBC0ATji003465@ip-10-146-233-104.ec2.internal>
Date: Tue, 12 Dec 2017 00:10:29 +0000
From: "Vuk Ercegovac (Code Review)" <gerrit@cloudera.org>
To: Lars Volker <lv@cloudera.com>, impala-cr@cloudera.com,
        reviews@impala.incubator.apache.org
Subject: =?UTF-8?Q?=5BImpala-ASF-CR=5D_IMPALA-4993=3A_extend_dictionary_filtering_to_collections=0A?=
In-Reply-To: <gerrit.1512529480000.If3a2abcfc3d0f7d18756816659fed77ce12668dd@gerrit.cloudera.org>
References: <gerrit.1512529480000.If3a2abcfc3d0f7d18756816659fed77ce12668dd@gerrit.cloudera.org>
Reply-To: vercegovac@cloudera.com, impala-cr@cloudera.com, lv@cloudera.com,
        marcelk@gmail.com, reviews@impala.incubator.apache.org
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Disposition: inline
User-Agent: Gerrit/2.14.2
Content-Type: multipart/alternative; boundary="LyIkkRwPprY="; charset=UTF-8
archived-at: Tue, 12 Dec 2017 01:02:00 -0000

--LyIkkRwPprY=
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hello Lars Volker, 

I'd like you to reexamine a change=2E Please visit

  =
  http://gerrit=2Ecloudera=2Eorg:8080/8775

to look at the new patch set (#=
3)=2E

Change subject: IMPALA-4993: extend dictionary filtering to collecti=
ons
=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=
=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=
=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E

IMPALA-=
4993: extend dictionary filtering to collections

Currently, top-level scal=
ar columns in parquet files can
be used at runtime to prune row-groups by e=
valuating certain
conjuncts over the column's dictionary (if available)=2E
=

This change extends such pruning to scalar values that are
stored in colle=
ction type columns=2E Currently, dictionary
pruning works by finding eligib=
le conjuncts for top-level
slots=2E Since only top-level slots are supporte=
d, the slots
are implicitly part of the scan node's tuple descriptor=2E
Wit=
h this change, we track eligible conjuncts by slot as well
as the tuple tha=
t contains the slot (either top-level or
nested collection)=2E Since collec=
tion conjuncts are already
managed by a map that associates tuple descripto=
rs to a list
of their conjuncts, this extension follows the existing
repres=
entation=2E

The frontend builds the mapping of <TupleId, SlotId> to
conjun=
cts that are dictionary filterable=2E The backend is
adjusted to use the sa=
me representation=2E In addition, collection
readers are decomposed into sc=
alar filterable columns and
other, non-dictionary filterable readers=2E Whe=
n filtering
a row group using a conjunct associated to a (possibly)
nested =
collection type, an additional tuple buffer is
allocated per tuple descript=
or=2E

Testing:
- e2e test extended to illustrate row-groups that are prune=
d
  by nested collection dictionary filters=2E

Change-Id: If3a2abcfc3d0f7d=
18756816659fed77ce12668dd
---
M be/src/exec/hdfs-parquet-scanner=2Ecc
M be/=
src/exec/hdfs-parquet-scanner=2Eh
M be/src/exec/hdfs-scan-node-base=2Ecc
M =
be/src/exec/hdfs-scan-node-base=2Eh
M be/src/exec/hdfs-scanner=2Ecc
M be/sr=
c/exec/hdfs-scanner=2Eh
M be/src/runtime/collection-value-builder=2Eh
M com=
mon/thrift/PlanNodes=2Ethrift
M fe/src/main/java/org/apache/impala/planner/=
HdfsScanNode=2Ejava
M testdata/workloads/functional-planner/queries/Planner=
Test/parquet-filtering=2Etest
M testdata/workloads/functional-query/queries=
/QueryTest/parquet-filtering=2Etest
11 files changed, 638 insertions(+), 19=
4 deletions(-)


  git pull ssh://gerrit=2Ecloudera=2Eorg:29418/Impala-ASF =
refs/changes/75/8775/3
-- 
To view, visit http://gerrit=2Ecloudera=2Eorg:80=
80/8775
To unsubscribe, visit http://gerrit=2Ecloudera=2Eorg:8080/settings
=

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpa=
tchset
Gerrit-Change-Id: If3a2abcfc3d0f7d18756816659fed77ce12668dd
Gerrit-C=
hange-Number: 8775
Gerrit-PatchSet: 3
Gerrit-Owner: Vuk Ercegovac <vercegov=
ac@cloudera=2Ecom>
Gerrit-Reviewer: Lars Volker <lv@cloudera=2Ecom>
Gerrit-=
Reviewer: Vuk Ercegovac <vercegovac@cloudera=2Ecom>

--LyIkkRwPprY=--