impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dan Hecht (Code Review)" <>
Subject [Impala-ASF-CR] IMPALA-5890: Abort queries if scanner hits IO errors
Date Mon, 11 Sep 2017 17:26:11 GMT
Dan Hecht has posted comments on this change.

Change subject: IMPALA-5890: Abort queries if scanner hits IO errors

Patch Set 5:

File be/src/exec/

PS5, Line 177:     if (status.IsCancelled() || status.IsMemLimitExceeded()) return status;
             :     // Log error from file format parsing.
             :     state_->LogError(ErrorMsg(TErrorCode::SEQUENCE_SCANNER_PARSE_ERROR,
             :         stream_->filename(), stream_->file_offset(),
             :         (stream_->eof() ? "(EOF)" : "")));
             :     // Make sure errors specified in the status are logged as well
             :     state_->LogError(status.msg());
             :     // If abort on error then return, otherwise try to recover.
             :     if (state_->abort_on_error()) return status;
             :     // Abort scan range for I/O related errors
             :     if (status.code() == TErrorCode::SCANNER_CONTEXT_WRAPPED_IO_ERROR) {
             :       eos_ = true;
             :       return Status::OK();
             :     }
it does seem like this could be written in terms of RuntimeState::LogOrReturnErrror(), right?
I guess the one complication is that we're trying not to fail entirely, but instead mark eos_
in that case. But are we sure that returning error from GetNextInternal() won't do the same
thing? I thought the scanners have a check higher up the callstack that would go ahead to
the next scan range if abort_on_error==false and GetNext() returns (recoverable) error, but
I could be mistaken.
File be/src/exec/

PS5, Line 194: Internal "
would be best to keep "Internal error" on the same line so that it's more easily grep-able.
File be/src/exec/scanner-context.h:

Line 277:     Status WrapIoError(const Status& status);
rather than wrapping, is it not possible to make our IO code return a single (or some set
of) unrecoverable IO error codes directly, that we can then identify as IO errors?

To view, visit
To unsubscribe, visit

Gerrit-MessageType: comment
Gerrit-Change-Id: I44dc95184c241fbcdbdbebad54339530680d3509
Gerrit-PatchSet: 5
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Lars Volker <>
Gerrit-Reviewer: Dan Hecht <>
Gerrit-Reviewer: Joe McDonnell <>
Gerrit-Reviewer: Lars Volker <>
Gerrit-Reviewer: Tim Armstrong <>
Gerrit-HasComments: Yes

View raw message