hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sushanth Sowmyan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-12303) HCatRecordSerDe throw a IndexOutOfBoundsException
Date Fri, 30 Oct 2015 16:40:27 GMT

    [ https://issues.apache.org/jira/browse/HIVE-12303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14982835#comment-14982835
] 

Sushanth Sowmyan commented on HIVE-12303:
-----------------------------------------

Can you provide the following info:

a) The schema of the table in hive, and what type of storage is used.
b) The schema of partitions (if any) in hive.
c) What pig says the schema is, when you do a describe of that table.

Basically, HCatRecordSerDe uses the size it's told. If it's told that there are > 24 columns
or elements in a struct, it'll try to access them, and in this case, fail since only 24 items
were present in the raw data beneath. So I would compare the data with the schema first in
this case.

>  HCatRecordSerDe  throw a IndexOutOfBoundsException 
> ----------------------------------------------------
>
>                 Key: HIVE-12303
>                 URL: https://issues.apache.org/jira/browse/HIVE-12303
>             Project: Hive
>          Issue Type: Bug
>          Components: HCatalog
>    Affects Versions: 0.14.0, 1.2.1
>            Reporter: Xiaowei Wang
>            Assignee: Sushanth Sowmyan
>             Fix For: 1.2.1
>
>         Attachments: HIVE-12303.0.patch
>
>
> When access hive table using hcatlog in Pig,sometime it throws a exception !
> Exception
> {noformat}
> 2015-10-30 06:44:35,219 WARN [Thread-4] org.apache.hadoop.mapred.YarnChild: Exception
running child : org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error converting
read value to tuple
>         at org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:76)
>         at org.apache.hive.hcatalog.pig.HCatLoader.getNext(HCatLoader.java:59)
>         at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:204)
>         at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
>         at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
>         at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
>         at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1892)
>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
> Caused by: java.lang.IndexOutOfBoundsException: Index: 24, Size: 24
>         at java.util.ArrayList.rangeCheck(ArrayList.java:635)
>         at java.util.ArrayList.get(ArrayList.java:411)
>         at org.apache.hive.hcatalog.data.HCatRecordSerDe.serializeStruct(HCatRecordSerDe.java:175)
>         at org.apache.hive.hcatalog.data.HCatRecordSerDe.serializeList(HCatRecordSerDe.java:244)
>         at org.apache.hive.hcatalog.data.HCatRecordSerDe.serializeField(HCatRecordSerDe.java:196)
>         at org.apache.hive.hcatalog.data.LazyHCatRecord.get(LazyHCatRecord.java:53)
>         at org.apache.hive.hcatalog.data.LazyHCatRecord.get(LazyHCatRecord.java:97)
>         at org.apache.hive.hcatalog.mapreduce.HCatRecordReader.nextKeyValue(HCatRecordReader.java:204)
>         at org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:63)
>         ... 13 more
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message