hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HIVE-16671) LLAP IO: BufferUnderflowException may happen in very rare(?) cases due to ORC end-of-CB estimation
Date Tue, 16 May 2017 00:28:05 GMT

    [ https://issues.apache.org/jira/browse/HIVE-16671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011544#comment-16011544
] 

Sergey Shelukhin edited comment on HIVE-16671 at 5/16/17 12:28 AM:
-------------------------------------------------------------------

[~prasanth_j] can you take a look? and also perhaps comment if you have input on ORC estimates.
This is basically the same logic as just below where we find out that BufferChunk-s from disk
contain a cut off CB, after finding the CB length. What we see is a stable repro of BufferUnderflow
in one of the 3 get-s. So I'm assuming that ORC estimate can overshoot a CB boundary by exactly
1-2 bytes, causing the failure to read length; that needs to be handled similarly to a partial
CB. Does this make sense?


was (Author: sershe):
[~prasanth_j] can you take a look? and also perhaps comment if you have input on ORC estimates.
This is basically the same logic as just below where we find out that BufferChunk-s from disk
contain a cut off CB, after finding the chunk length. What we see is a stable repro of BufferUnderflow
in one of the 3 get-s. So I'm assuming that ORC estimate can overshoot a CB by exactly 1-2
bytes, causing the failure to read length; that needs to be handled similarly to a partial
chunk. Does this make sense?

> LLAP IO: BufferUnderflowException may happen in very rare(?) cases due to ORC end-of-CB
estimation
> --------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-16671
>                 URL: https://issues.apache.org/jira/browse/HIVE-16671
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Ravi Mutyala
>            Assignee: Sergey Shelukhin
>         Attachments: HIVE-16671.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message