hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Dere (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-16637) Improve end-of-data checking for LLAP input format
Date Wed, 10 May 2017 21:10:04 GMT

    [ https://issues.apache.org/jira/browse/HIVE-16637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16005425#comment-16005425
] 

Jason Dere commented on HIVE-16637:
-----------------------------------

Whoops, forgot to link the RB: https://reviews.apache.org/r/59152/
In the existing logic, LlapBaseRecordReader would just go ahead and try to read from the DataInputStream,
regardless of whether the underlying stream was finished or not. This does not work - methods
like WritableUtils.readVInt() assume there is sufficient input in the stream and will throw
EOFException if there is not. Anyway, we should never have ignored EOFException - that was
not correct. We should never try to initiate a read if the stream is empty, thus adding the
hasInput() check. I believe the mark/reset usage is correct - I think that's the point of
having those methods.


> Improve end-of-data checking for LLAP input format
> --------------------------------------------------
>
>                 Key: HIVE-16637
>                 URL: https://issues.apache.org/jira/browse/HIVE-16637
>             Project: Hive
>          Issue Type: Bug
>          Components: llap
>            Reporter: Jason Dere
>            Assignee: Jason Dere
>         Attachments: HIVE-16637.1.patch
>
>
> The existing end of stream checking in the record reader is too forgiving of errors and
does not recognize situations where the server connection has closed abruptly like HIVE-14093.
> Try to add a way to indicate that we have truly hit the end of the stream.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message