hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phabricator (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-4494) ORC map columns get class cast exception in some context
Date Mon, 06 May 2013 23:48:16 GMT

    [ https://issues.apache.org/jira/browse/HIVE-4494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13650248#comment-13650248
] 

Phabricator commented on HIVE-4494:
-----------------------------------

pamelavagata has commented on the revision "HIVE-4494 [jira] ORC map columns get class cast
exception in some context".

  Owen, I had previously investigated this issue and found that the reason for the ClassCastException
was due to the following reason:

  in ObjectInspectorConverters.getConverter(ObjectInspector inputOI, ObjectInspector outputOI),
  there's a referential equality check to see if the inputOI is the same as the outputOI and
returns a new Identity Converter. When running the same query with RC, you'll find that ColumnarSerde
caches the the object inspectors and reuses them, which is why the referential equality check
will pass and return an IdentityConverter instead of trying to cast to a Settable*ObjectInspector.

  If you cache the object inspectors in OrcStruct instead of having OrcList/OrcMap object
inspectors inherit from Settable*ObjectInspector, it will fix the underlying issue. I hadn't
gotten around to submitting a diff for this yet but would be happy to push one out in the
next 2 days.

REVISION DETAIL
  https://reviews.facebook.net/D10653

To: JIRA, omalley
Cc: pamelavagata

                
> ORC map columns get class cast exception in some context
> --------------------------------------------------------
>
>                 Key: HIVE-4494
>                 URL: https://issues.apache.org/jira/browse/HIVE-4494
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Owen O'Malley
>            Assignee: Owen O'Malley
>         Attachments: HIVE-4494.D10653.1.patch
>
>
> Setting up the test case like:
> {quote}
> create table map_text (
>   name string,
>   m map<string,string>
> ) row format delimited
>     fields terminated by '|'
>     collection items terminated by ','
>     map keys terminated by ':';
> create table map_orc (
>   name string,
>   m map<string,string>
> ) stored as orc;
> cat map.txt
> name1|key11:value11,key12:value12,key13:value13
> name2|key21:value21,key22:value22,key23:value23
> name3|key31:value31,key32:value32,key33:value33
> load data local	inpath 'map.txt' into table map_text;
> insert overwrite table map_orc select * from map_text;
> {quote}
> Selecting the name column from orc_map will get the following exception:
> {quote}
> java.lang.RuntimeException: Error in configuring object
> 	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
> 	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
> 	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> 	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:431)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:371)
> 	at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:396)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1195)
> 	at org.apache.hadoop.mapred.Child.main(Child.java:249)
> Caused by: java.lang.reflect.InvocationTargetException
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
> 	... 9 more
> Caused by: java.lang.RuntimeException: Error in configuring object
> 	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
> 	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
> 	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> 	at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> 	... 14 more
> Caused by: java.lang.reflect.InvocationTargetException
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
> 	... 17 more
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> 	at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> 	... 22 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassCastException:
org.apache.hadoop.hive.ql.io.orc.OrcStruct$OrcMapObjectInspector cannot be cast to org.apache.hadoop.hive.serde2.objectinspector.SettableMapObjectInspector
> 	at org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:522)
> 	at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:90)
> 	... 22 more
> Caused by: java.lang.ClassCastException: org.apache.hadoop.hive.ql.io.orc.OrcStruct$OrcMapObjectInspector
cannot be cast to org.apache.hadoop.hive.serde2.objectinspector.SettableMapObjectInspector
> 	at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters.getConverter(ObjectInspectorConverters.java:144)
> 	at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters$StructConverter.<init>(ObjectInspectorConverters.java:307)
> 	at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters.getConverter(ObjectInspectorConverters.java:138)
> 	at org.apache.hadoop.hive.ql.exec.MapOperator.initObjectInspector(MapOperator.java:270)
> 	at org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:482)
> 	... 23 more
> {quote}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message