impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Impala Public Jenkins (Code Review)" <>
Subject [Impala-ASF-CR] IMPALA-5197: Erroneous corrupted Parquet file message
Date Tue, 09 May 2017 09:27:40 GMT
Impala Public Jenkins has submitted this change and it was merged.

Change subject: IMPALA-5197: Erroneous corrupted Parquet file message

IMPALA-5197: Erroneous corrupted Parquet file message

The Parquet file column reader may fail in the middle
of producing a scratch tuple batch for various reasons
such as exceeding memory limit or cancellation. In which
case, the scratch tuple batch may not have materialized
all the rows in a row group. We shouldn't erroneously
report that the file is corrupted in this case as the
column reader didn't completely read the entire row group.

A new test case is added to verify that we won't see this
error message. A new failpoint phase GETNEXT_SCANNER is
also added to differentiate it from the GETNEXT in the
scan node itself.

Change-Id: I9138039ec60fbe9deff250b8772036e40e42e1f6
Reviewed-by: Michael Ho <>
Tested-by: Impala Public Jenkins
M be/src/exec/
M be/src/exec/
M be/src/exec/hdfs-scan-node-base.h
M be/src/exec/hdfs-scanner.h
M be/src/exec/
M be/src/exec/parquet-column-readers.h
M common/thrift/PlanNodes.thrift
M tests/common/
M tests/failure/
9 files changed, 40 insertions(+), 24 deletions(-)

  Impala Public Jenkins: Verified
  Michael Ho: Looks good to me, approved

To view, visit
To unsubscribe, visit

Gerrit-MessageType: merged
Gerrit-Change-Id: I9138039ec60fbe9deff250b8772036e40e42e1f6
Gerrit-PatchSet: 7
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Michael Ho <>
Gerrit-Reviewer: Alex Behm <>
Gerrit-Reviewer: Dan Hecht <>
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Michael Ho <>
Gerrit-Reviewer: Tim Wood <>

View raw message