hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prasanth Jayachandran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-18488) LLAP ORC readers are missing some null checks
Date Thu, 18 Jan 2018 20:01:01 GMT

    [ https://issues.apache.org/jira/browse/HIVE-18488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16331100#comment-16331100
] 

Prasanth Jayachandran commented on HIVE-18488:
----------------------------------------------

+1.
I am guessing the tree readers can be null when there is schema evolution. When reader schema
has more fields than file schema there can be nulls in the readers for the missing columns.


> LLAP ORC readers are missing some null checks
> ---------------------------------------------
>
>                 Key: HIVE-18488
>                 URL: https://issues.apache.org/jira/browse/HIVE-18488
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Deepesh Khandelwal
>            Assignee: Sergey Shelukhin
>            Priority: Major
>         Attachments: HIVE-18488.patch
>
>
> All the uses of StructTreeReader fields in ORCs have null checks; these are missing in
LLAP readers
> {noformat}
> Caused by: java.lang.NullPointerException
>                 at org.apache.hadoop.hive.ql.io.orc.encoded.EncodedTreeReaderFactory$StructStreamReader.setBuffers(EncodedTreeReaderFactory.java:2743)
>                 at org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.repositionInStreams(OrcEncodedDataConsumer.java:251)
>                 at org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.decodeBatch(OrcEncodedDataConsumer.java:138)
>                 at org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.decodeBatch(OrcEncodedDataConsumer.java:62)
>                 at org.apache.hadoop.hive.llap.io.decode.EncodedDataConsumer.consumeData(EncodedDataConsumer.java:81)
>                 at org.apache.hadoop.hive.llap.io.decode.EncodedDataConsumer.consumeData(EncodedDataConsumer.java:34)
>                 at org.apache.hadoop.hive.ql.io.orc.encoded.EncodedReaderImpl.readEncodedColumns(EncodedReaderImpl.java:482)
>                 at org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.performDataRead(OrcEncodedDataReader.java:417)
>                 at org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader$4.run(OrcEncodedDataReader.java:276)
>                 at org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader$4.run(OrcEncodedDataReader.java:273)
>                 at java.security.AccessController.doPrivileged(Native Method)
>                 at javax.security.auth.Subject.doAs(Subject.java:422)
>                 at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1965)
>                 at org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:273)
>                 at org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:110)
>                 ... 6 more
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message