impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gabor Kaszab (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-4826: Fix error during a scan on repeated root schema in Parquet.
Date Thu, 31 Aug 2017 15:06:08 GMT
Hello Tim Armstrong,

I'd like you to reexamine a change.  Please visit

    http://gerrit.cloudera.org:8080/7870

to look at the new patch set (#3).

Change subject: IMPALA-4826: Fix error during a scan on repeated root schema in Parquet.
......................................................................

IMPALA-4826: Fix error during a scan on repeated root schema in Parquet.

Having the repetition level set to REPEATED on the root schema
resulted a scan to fail with error when Impala tried to parse that
table.

As a solution, the 'REPEATED' repetition level is ignored when the
root schema is processed. The reasoning behind is that the Parquet
format description says that the repetition level of the root schema
should not be set to REPEATED anyway, so it's safe to ignore it in
case it is set to this value for some reason.

Change-Id: I7ea84589e1d122ad9d43adde46893ec0ecc5f9c4
---
M be/src/exec/parquet-metadata-utils.cc
M testdata/data/README
A testdata/data/repeated_root_schema.parquet
M tests/query_test/test_scanners.py
4 files changed, 27 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/70/7870/3
-- 
To view, visit http://gerrit.cloudera.org:8080/7870
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7ea84589e1d122ad9d43adde46893ec0ecc5f9c4
Gerrit-PatchSet: 3
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Gabor Kaszab <gaborkaszab@cloudera.com>
Gerrit-Reviewer: Alex Behm <alex.behm@cloudera.com>
Gerrit-Reviewer: Attila Jeges <attilaj@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <gaborkaszab@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <laszlo.gaal@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>

Mime
View raw message