hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <>
Subject [jira] [Commented] (HIVE-16637) Improve end-of-data checking for LLAP input format
Date Wed, 10 May 2017 22:24:04 GMT


Sergey Shelukhin commented on HIVE-16637:

I was suggesting that lengths 0 is passed at the end of the stream, indicating to Chunk reader
that it's end of stream. 
I was assuming that the caller can have better API to work with, i.e. smth like boolean next(OutputObject
o) { int l = readLength(stream); if (l == 0) return false; readFully(stream, l); o.readFrom(stream);
return true; } but I guess with writables it's not as easy.
+1 nit:
+      } catch (IOException err) {
+        throw new IOException("Unexpected EOF while attempting to read chunk length", err);
+      }
{noformat} why does it assume every IO exception is EOF? that's the other thing above.

> Improve end-of-data checking for LLAP input format
> --------------------------------------------------
>                 Key: HIVE-16637
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>          Components: llap
>            Reporter: Jason Dere
>            Assignee: Jason Dere
>         Attachments: HIVE-16637.1.patch
> The existing end of stream checking in the record reader is too forgiving of errors and
does not recognize situations where the server connection has closed abruptly like HIVE-14093.
> Try to add a way to indicate that we have truly hit the end of the stream.

This message was sent by Atlassian JIRA

View raw message