hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From vivek veeramani <vivek.veeraman...@gmail.com>
Subject Re: ORC Table creation failure
Date Thu, 21 May 2015 14:38:16 GMT
Hi Abe,

The column in the WHERE clause is also string. I tried creating without any
filters, just a CTAS and that failed too.

Thanks,
Vivek

On Thu, May 21, 2015 at 7:47 PM, Abe Weinograd <abe@flonet.com> wrote:

> I think your WHERE clause is having casting issues.  Is that column of the
> right data type?  Can you run the SELECT by itself without the CREATE TABLE?
>
> Abe
>
> On Wed, May 20, 2015 at 11:23 PM, vivek veeramani <
> vivek.veeramani87@gmail.com> wrote:
>
>>
>> Hi ,
>>
>> I'm trying to create a subset table from a master table which is stored
>> in ORC format. The query I'm using is :
>>
>>  CREATE TABLE tab_20150510 STORED AS ORC AS SELECT * FROM master_tab
>> WHERE col_id_60 <> '2015-05-20';
>>
>> The master table is a table with 60 columns with all string data type but
>> 1 column as bigint which is not part of the query. I tried creating this
>> using Tez as well but no go.
>> The error I get is shown below:
>>
>> -----
>> Diagnostic Messages for this Task:
>> Error: java.io.IOException: java.io.IOException:
>> java.lang.ClassCastException: org.apache.hadoop.io.IntWritable cannot be
>> cast to org.apache.hadoop.io.Text
>>         at
>> org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderNextException(HiveIOExceptionHandlerChain.java:121)
>>         at
>> org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderNextException(HiveIOExceptionHandlerUtil.java:77)
>>         at
>> org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.doNextWithExceptionHandler(HadoopShimsSecure.java:256)
>>         at
>> org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.next(HadoopShimsSecure.java:171)
>>         at
>> org.apache.hadoop.mapred.MapTask$TrackedRecordReader.moveToNext(MapTask.java:198)
>>         at
>> org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:184)
>>         at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:52)
>>         at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
>>         at java.security.AccessController.doPrivileged(Native Method)
>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>         at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
>> Caused by: java.io.IOException: java.lang.ClassCastException:
>> org.apache.hadoop.io.IntWritable cannot be cast to org.apache.hadoop.io.Text
>>         at
>> org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderNextException(HiveIOExceptionHandlerChain.java:121)
>>         at
>> org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderNextException(HiveIOExceptionHandlerUtil.java:77)
>>         at
>> org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:344)
>>         at
>> org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.doNext(CombineHiveRecordReader.java:101)
>>         at
>> org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.doNext(CombineHiveRecordReader.java:41)
>>         at
>> org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.next(HiveContextAwareRecordReader.java:122)
>>         at
>> org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.doNextWithExceptionHandler(HadoopShimsSecure.java:254)
>>         ... 11 more
>> Caused by: java.lang.ClassCastException: org.apache.hadoop.io.IntWritable
>> cannot be cast to org.apache.hadoop.io.Text
>>         at
>> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl$StringDictionaryTreeReader.next(RecordReaderImpl.java:1592)
>>         at
>> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl$StringTreeReader.next(RecordReaderImpl.java:1346)
>>         at
>> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl$StructTreeReader.next(RecordReaderImpl.java:1788)
>>         at
>> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.next(RecordReaderImpl.java:2997)
>>         at
>> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$OrcRecordReader.next(OrcInputFormat.java:153)
>>         at
>> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$OrcRecordReader.next(OrcInputFormat.java:127)
>>         at
>> org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:339)
>>         ... 15 more
>>
>> Container killed by the ApplicationMaster.
>> Container killed on request. Exit code is 143
>> Container exited with a non-zero exit code 143
>>
>>
>> FAILED: Execution Error, return code 2 from
>> org.apache.hadoop.hive.ql.exec.mr.MapRedTask
>> MapReduce Jobs Launched:
>> Job 0: Map: 1544   Cumulative CPU: 250716.7 sec   HDFS Read: 160494367819
>> HDFS Write: 121322793549 FAIL
>> Total MapReduce CPU Time Spent: 2 days 21 hours 38 minutes 36 seconds 700
>> msec
>>
>>
>> Greatly appreciate any help or information on this and how to rectify it.
>>
>>
>> --
>> Thanks ,
>> Vivek Veeramani
>>
>>
>> cell : +91-9632 975 975
>>         +91-9895 277 101
>>
>
>


-- 
Thanks ,
Vivek Veeramani


cell : +91-9632 975 975
        +91-9895 277 101

Mime
View raw message