hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Dere (JIRA)" <>
Subject [jira] [Commented] (HIVE-16637) Improve end-of-data checking for LLAP input format
Date Wed, 10 May 2017 21:10:04 GMT


Jason Dere commented on HIVE-16637:

Whoops, forgot to link the RB:
In the existing logic, LlapBaseRecordReader would just go ahead and try to read from the DataInputStream,
regardless of whether the underlying stream was finished or not. This does not work - methods
like WritableUtils.readVInt() assume there is sufficient input in the stream and will throw
EOFException if there is not. Anyway, we should never have ignored EOFException - that was
not correct. We should never try to initiate a read if the stream is empty, thus adding the
hasInput() check. I believe the mark/reset usage is correct - I think that's the point of
having those methods.

> Improve end-of-data checking for LLAP input format
> --------------------------------------------------
>                 Key: HIVE-16637
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>          Components: llap
>            Reporter: Jason Dere
>            Assignee: Jason Dere
>         Attachments: HIVE-16637.1.patch
> The existing end of stream checking in the record reader is too forgiving of errors and
does not recognize situations where the server connection has closed abruptly like HIVE-14093.
> Try to add a way to indicate that we have truly hit the end of the stream.

This message was sent by Atlassian JIRA

View raw message