hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HIVE-9979) LLAP: LLAP Cached readers for StringDirectTreeReaders over-read data
Date Thu, 19 Mar 2015 02:25:38 GMT

     [ https://issues.apache.org/jira/browse/HIVE-9979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sergey Shelukhin resolved HIVE-9979.
------------------------------------
       Resolution: Fixed
    Fix Version/s: llap

> LLAP: LLAP Cached readers for StringDirectTreeReaders over-read data
> --------------------------------------------------------------------
>
>                 Key: HIVE-9979
>                 URL: https://issues.apache.org/jira/browse/HIVE-9979
>             Project: Hive
>          Issue Type: Sub-task
>    Affects Versions: llap
>            Reporter: Gopal V
>            Assignee: Sergey Shelukhin
>             Fix For: llap
>
>         Attachments: HIVE-9979.patch
>
>
> When the cache is enabled, queries throws different over-read exceptions.
> Looks like the batchSize changes as you read data, the end of stripe batchSize is smaller
than the default size (the super calls change it).
> {code}
> Caused by: java.io.EOFException: Can't finish byte read from uncompressed stream DATA
position: 262144 length: 262144 range: 0 offset: 46399488 limit: 46399488
>         at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl$BytesColumnVectorUtil.commonReadByteArrays(RecordReaderImpl.java:1556)
>         at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl$BytesColumnVectorUtil.readOrcByteArrays(RecordReaderImpl.java:1569)
>         at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl$StringDirectTreeReader.nextVector(RecordReaderImpl.java:1691)
>         at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl$StringTreeReader.nextVector(RecordReaderImpl.java:1517)
>         at org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.decodeBatch(OrcEncodedDataConsumer.java:115)
>         at org.apache.hadoop.hive.llap.io.decode.EncodedDataConsumer.consumeData(EncodedDataConsumer.java:108)
>         at org.apache.hadoop.hive.llap.io.decode.EncodedDataConsumer.consumeData(EncodedDataConsumer.java:35)
>         at org.apache.hadoop.hive.ql.io.orc.EncodedReaderImpl.readEncodedColumns(EncodedReaderImpl.java:314)
>         at org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:280)
>         at org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:44)
>         at org.apache.hadoop.hive.common.CallableWithNdc.call(CallableWithNdc.java:37)
>         ... 4 more
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message