impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Armstrong (Code Review)" <>
Subject [Impala-ASF-CR] IMPALA-5890: Abort queries if scanner hits IO errors
Date Sat, 09 Sep 2017 00:20:59 GMT
Tim Armstrong has posted comments on this change.

Change subject: IMPALA-5890: Abort queries if scanner hits IO errors

Patch Set 3:

Commit Message:

Line 19: - Then during error handling, the BaseSequenceScanner calls SkipToSync()
I wonder if the same bug can happen for text files.

I think parquet may be OK since it gives up on the row group when it encounters an error.
File be/src/exec/

PS3, Line 187: Abort
I'm wondering whether we should abort the whole query or abort scanning this file only by
setting eos_. The second seems more in the spirit of abort_on_error=0.
File be/src/exec/

Line 156:     Status status(scan_range_->GetNext(&io_buffer_));
  Status status = scan_range_->GetNext(...)

I guess it's equivalent but I had to read it twice since it's different from the normal idiom.

Line 193:     return Status(TErrorCode::GENERAL_IO_ERROR, Substitute("Failed to receive buffer
This would be really confusing if a user hit this. I think we should make it clear that it's
an internal error from the message.

We should also file a JIRA to remove this, since we don't want this workaround hanging around

Line 364:   if (status.ok() || status.code() == TErrorCode::GENERAL_IO_ERROR) return status;
Can we file a follow-on JIRA to rework this - e.g. ensure that all HDFS read errors coming
out of the I/O manager are identifiable.

Line 365:   Status wrapped_status(TErrorCode::GENERAL_IO_ERROR);
Does the wrapped error show up in the error log and query profile ok? I know there's some
weirdness with merged errors getting dropped in some cases: IMPALA-4697
File common/thrift/

Line 337:   ("GENERAL_IO_ERROR", 110, "General I/O error."),
Can we call this something like SCANNER_CONTEXT_WRAPPED_IO_ERROR for now? It's a bit confusing
since it sounds like it's more general than it is.

To view, visit
To unsubscribe, visit

Gerrit-MessageType: comment
Gerrit-Change-Id: I44dc95184c241fbcdbdbebad54339530680d3509
Gerrit-PatchSet: 3
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Lars Volker <>
Gerrit-Reviewer: Joe McDonnell <>
Gerrit-Reviewer: Lars Volker <>
Gerrit-Reviewer: Tim Armstrong <>
Gerrit-HasComments: Yes

View raw message