impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Behm (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-3943: Adhere to abort on error in ProcessFooter().
Date Wed, 17 Aug 2016 06:00:11 GMT
Hello Tim Armstrong,

I'd like you to reexamine a change.  Please visit

    http://gerrit.cloudera.org:8080/3862

to look at the new patch set (#5).

Change subject: IMPALA-3943: Adhere to abort_on_error in ProcessFooter().
......................................................................

IMPALA-3943: Adhere to abort_on_error in ProcessFooter().

This patch focuses on properly handling abort_on_error when ProcessFooter()
fails in the Parquet scanner.
It also refactors a few existing tests to cover both possible values
of abort_on_error.

There are still several places left where abort_on_error is not honored.

Testing: I ran test_scanners.py::TestParquet and test_stale_metadata.py
locally on exhaustive with debug and asan builds. I ran the data errors
tests in a loop for 20 iterations. I also did a private core/hdfs run.

Change-Id: I6aff766a1ce6376efb329bdde51c648149dfe08c
---
M be/src/exec/hdfs-parquet-scanner.cc
M be/src/exec/hdfs-parquet-scanner.h
M testdata/data/README
A testdata/data/no_row_groups.parquet
M testdata/workloads/functional-query/queries/DataErrorsTest/avro-errors.test
M testdata/workloads/functional-query/queries/DataErrorsTest/hdfs-scan-node-errors.test
A testdata/workloads/functional-query/queries/QueryTest/no-row-groups-abort.test
A testdata/workloads/functional-query/queries/QueryTest/no-row-groups.test
M testdata/workloads/functional-query/queries/QueryTest/parquet-abort-on-error.test
M testdata/workloads/functional-query/queries/QueryTest/parquet-continue-on-error.test
M testdata/workloads/functional-query/queries/QueryTest/parquet.test
M tests/common/test_dimensions.py
M tests/common/test_result_verifier.py
M tests/metadata/test_stale_metadata.py
M tests/query_test/test_scanners.py
15 files changed, 176 insertions(+), 78 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/62/3862/5
-- 
To view, visit http://gerrit.cloudera.org:8080/3862
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I6aff766a1ce6376efb329bdde51c648149dfe08c
Gerrit-PatchSet: 5
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Alex Behm <alex.behm@cloudera.com>
Gerrit-Reviewer: Alex Behm <alex.behm@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhecht@cloudera.com>
Gerrit-Reviewer: Matthew Jacobs <mj@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>

Mime
View raw message