Mailing-List: contact reviews-help@impala.apache.org; run by ezmlm
Precedence: bulk
Message-Id: <201711221804.vAMI4TSk001786@ip-10-146-233-104.ec2.internal>
Date: Wed, 22 Nov 2017 18:04:28 +0000
From: "Vuk Ercegovac (Code Review)" <gerrit@cloudera.org>
To: Lars Volker <lv@cloudera.com>, Alex Behm <alex.behm@cloudera.com>,
        impala-cr@cloudera.com, reviews@impala.incubator.apache.org
Subject: =?UTF-8?Q?=5BImpala-ASF-CR=5D_IMPALA-4985=3A_use_parquet_stats_of_nested_types_for_dynamic_pruning=0A?=
In-Reply-To: <gerrit.1509953504000.I0c99e20cb080b504442cd5376ea3e046016158fe@gerrit.cloudera.org>
References: <gerrit.1509953504000.I0c99e20cb080b504442cd5376ea3e046016158fe@gerrit.cloudera.org>
Reply-To: vercegovac@cloudera.com, impala-cr@cloudera.com, lv@cloudera.com,
        marcelk@gmail.com, alex.behm@cloudera.com,
        reviews@impala.incubator.apache.org
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Disposition: inline
User-Agent: Gerrit/2.14.2
Content-Type: multipart/alternative; boundary="mVPu6CD/J6M="; charset=UTF-8
archived-at: Wed, 22 Nov 2017 18:04:34 -0000

--mVPu6CD/J6M=
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hello Lars Volker, Alex Behm, Impala Public Jenkins, 

I'd like you to reex=
amine a change=2E Please visit

    http://gerrit=2Ecloudera=2Eorg:8080/848=
0

to look at the new patch set (#10)=2E

Change subject: IMPALA-4985: use =
parquet stats of nested types for dynamic pruning
=2E=2E=2E=2E=2E=2E=2E=2E=
=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=
=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=
=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E

IMPALA-4985: use parquet stats of nes=
ted types for dynamic pruning

Currently, parquet row-groups can be pruned =
at run-time using
min/max stats when predicates (in, binary) are specified =
for
column scalar types=2E This patch extends pruning to nested types
for t=
he same class of predicates=2E A nested value is an instance
of a nested ty=
pe (struct, array, map)=2E A nested value consists of
other nested and scal=
ar values (as declared by its type)=2E
Predicates that can be used for row-=
group pruning must be applied to
nested scalar values=2E In addition, the p=
arent of the nested scalar
must also be required, that is, not empty=2E The=
 latter requirement
is conservative: some filters that could be used for pr=
uning are
not used for correctness reasons=2E

Testing:
- extended nested-t=
ypes-parquet-stats e2e test cases=2E

Change-Id: I0c99e20cb080b504442cd5376=
ea3e046016158fe
---
M be/src/exec/hdfs-parquet-scanner=2Eh
M fe/src/main/ja=
va/org/apache/impala/analysis/CollectionStructType=2Ejava
M fe/src/main/jav=
a/org/apache/impala/analysis/SelectStmt=2Ejava
M fe/src/main/java/org/apach=
e/impala/analysis/SlotRef=2Ejava
M fe/src/main/java/org/apache/impala/plann=
er/HdfsScanNode=2Ejava
M testdata/workloads/functional-planner/queries/Plan=
nerTest/constant-folding=2Etest
M testdata/workloads/functional-planner/que=
ries/PlannerTest/mt-dop-validation=2Etest
M testdata/workloads/functional-p=
lanner/queries/PlannerTest/parquet-filtering=2Etest
M testdata/workloads/fu=
nctional-query/queries/QueryTest/nested-types-parquet-stats=2Etest
M tests/=
query_test/test_nested_types=2Epy
10 files changed, 649 insertions(+), 38 d=
eletions(-)


  git pull ssh://gerrit=2Ecloudera=2Eorg:29418/Impala-ASF ref=
s/changes/80/8480/10
-- 
To view, visit http://gerrit=2Ecloudera=2Eorg:8080=
/8480
To unsubscribe, visit http://gerrit=2Ecloudera=2Eorg:8080/settings

G=
errit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatc=
hset
Gerrit-Change-Id: I0c99e20cb080b504442cd5376ea3e046016158fe
Gerrit-Cha=
nge-Number: 8480
Gerrit-PatchSet: 10
Gerrit-Owner: Vuk Ercegovac <vercegova=
c@cloudera=2Ecom>
Gerrit-Reviewer: Alex Behm <alex=2Ebehm@cloudera=2Ecom>
G=
errit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Lars Volker <lv@clou=
dera=2Ecom>
Gerrit-Reviewer: Vuk Ercegovac <vercegovac@cloudera=2Ecom>

--mVPu6CD/J6M=--