hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eugene Koifman (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-6724) HCatStorer throws ClassCastException while storing tinyint/smallint data
Date Fri, 21 Mar 2014 20:47:43 GMT

    [ https://issues.apache.org/jira/browse/HIVE-6724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13943524#comment-13943524
] 

Eugene Koifman commented on HIVE-6724:
--------------------------------------

The use case is Hive-Pig-Hive.
HCatLoader automatically sets hcat.data.tiny.small.int.promotion=true as that is required
by Pig. HCatStorer did the opposite for no good reason. Since Pig doesn't evaluate each statement
separately, the Storer action clobbered Loader action (the context that contains the configuration
is shared). I changed Storer not to do that.

> HCatStorer throws ClassCastException while storing tinyint/smallint data
> ------------------------------------------------------------------------
>
>                 Key: HIVE-6724
>                 URL: https://issues.apache.org/jira/browse/HIVE-6724
>             Project: Hive
>          Issue Type: Bug
>          Components: HCatalog
>    Affects Versions: 0.12.0
>            Reporter: Eugene Koifman
>            Assignee: Eugene Koifman
>
> given Hive tables:
> 1) create table pig_hcatalog_1 (si smallint)  STORED AS TEXTFILE;
> 2) create table all100k (si smallint, ti tinyint) STORED ....;
> the following sequence of steps (assuming there is data in all100k)
> {noformat}
> a=load 'all100k' using org.apache.hive.hcatalog.pig.HCatLoader();
> b = foreach a generate si;
> store b into 'pig_hcatalog_1' using org.apache.hive.hcatalog.pig.HCatStorer();
> {noformat}
> produces 
> {noformat}
> org.apache.hadoop.mapred.YarnChild: Exception running child : java.lang.ClassCastException:
java.lang.Short cannot be cast to java.lang.Integer
> 	at org.apache.hive.hcatalog.pig.HCatBaseStorer.getJavaObj(HCatBaseStorer.java:372)
> 	at org.apache.hive.hcatalog.pig.HCatBaseStorer.putNext(HCatBaseStorer.java:306)
> 	at org.apache.hive.hcatalog.pig.HCatStorer.putNext(HCatStorer.java:61)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:139)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:98)
> 	at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:635)
> 	at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
> 	at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.collect(PigMapOnly.java:48)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:284)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:277)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
> 	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
> 	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
> 	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:396)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
> 	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message