hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "@Sanjiv Singh" <sanjiv.is...@gmail.com>
Subject HiveException: Unexpected exception: org.apache.hadoop.io.Text cannot be cast to org.apache.hadoop.io.IntWritable
Date Mon, 11 Jan 2016 12:31:29 GMT
I am frequently facing casting error with Hive query. One sample error logs
given below. Error contains following details :


   - *complete row data  *

        HiveException: Hive Runtime Error while processing row
{"company_id":"3M","oper_carrier":"3M","flt_num":"4062","equip_id":"0331","equip_type":"SF3","act_fleet_type_cd":"U","shares_ship_id":"0331","report_dt":"2013-12-12","origin":"CKB","destination":"IAD"}


   - * costing exception*

        Caused by: java.lang.ClassCastException: org.apache.hadoop.io.Text
cannot be cast to org.apache.hadoop.io.IntWritable


I don't know how to debug such exception. It describes complete row , not
the value of a column. I already have checked on both the tables for values
and their data type.

Please help me identifying the issue and resolving it.


*Query ran :*

    SELECT
>
> vss.company_id,vss.shares_ship_id,vss.seatmap_cd,vss.cabin,vss.seat,
> vss.seat_loc_dscr, vss.ep_seat AS EPlus_Seat, vss.ep_win_seat,
> vss.ep_asle_seat
>     FROM rvsed11 zz
>     LEFT OUTER JOIN rvsed22 vss
>      ON zz.company_id = vss.company_id
>      AND zz.shares_ship_id = vss.shares_ship_id
>


*See complete log :*


    Query ID = root_20160111201839_bd6220b3-cca8-48ee-8d32-4e26fdaab8bf
    Total jobs = 1
    Launching Job 1 out of 1
    Number of reduce tasks is set to 0 since there's no reduce operator
    Starting Job = job_1450202704586_0448, Tracking URL =
http://server-D912:8088/proxy/application_1450202704586_0448/
    Kill Command = /usr/share/hadoop_ecosystem/hadoop-2.6.0/bin/hadoop job
-kill job_1450202704586_0448
    Hadoop job information for Stage-3: number of mappers: 1; number of
reducers: 0
    2016-01-11 20:18:43,960 Stage-3 map = 0%,  reduce = 0%
    2016-01-11 20:19:02,202 Stage-3 map = 100%,  reduce = 0%
    Ended Job = job_1450202704586_0448 with errors
    Error during job, obtaining debugging information...
    Examining task ID: task_1450202704586_0448_m_000000 (and more) from job
job_1450202704586_0448

    Task with the most failures(4):
    -----
    Task ID:
      task_1450202704586_0448_m_000000

    URL:

http://server-D912:8088/taskdetails.jsp?jobid=job_1450202704586_0448&tipid=task_1450202704586_0448_m_000000
    -----
    Diagnostic Messages for this Task:
    Error: java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row
{"company_id":"3M","oper_carrier":"3M","flt_num":"4062","equip_id":"0331","equip_type":"SF3","act_fleet_type_cd":"U","shares_ship_id":"0331","report_dt":"2013-12-12","origin":"CKB","destination":"IAD"}

    at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:172)
    at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
    Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive
Runtime Error while processing row
{"company_id":"3M","oper_carrier":"3M","flt_num":"4062","equip_id":"0331","equip_type":"SF3","act_fleet_type_cd":"U","shares_ship_id":"0331","report_dt":"2013-12-12","origin":"CKB","destination":"IAD"}
    at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:518)
    at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:163)
    ... 8 more
    Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unexpected
exception: **org.apache.hadoop.io.Text cannot be cast to
org.apache.hadoop.io.IntWritable**
    at
org.apache.hadoop.hive.ql.exec.MapJoinOperator.process(MapJoinOperator.java:426)
    at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:837)
    at
org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:97)
    at
org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:162)
    at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:508)
    ... 9 more
    Caused by: java.lang.ClassCastException: org.apache.hadoop.io.Text
cannot be cast to org.apache.hadoop.io.IntWritable
    at
org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableIntObjectInspector.get(WritableIntObjectInspector.java:36)
    at
org.apache.hadoop.hive.serde2.lazy.LazyUtils.writePrimitiveUTF8(LazyUtils.java:202)
    at
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.serialize(LazySimpleSerDe.java:307)
    at
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.serializeField(LazySimpleSerDe.java:262)
    at
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.doSerialize(LazySimpleSerDe.java:246)
    at
org.apache.hadoop.hive.serde2.AbstractEncodingAwareSerDe.serialize(AbstractEncodingAwareSerDe.java:50)
    at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:720)
    at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:837)
    at
org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:88)
    at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:837)
    at
org.apache.hadoop.hive.ql.exec.CommonJoinOperator.internalForward(CommonJoinOperator.java:644)
    at
org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genUniqueJoinObject(CommonJoinOperator.java:657)
    at
org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genUniqueJoinObject(CommonJoinOperator.java:660)
    at
org.apache.hadoop.hive.ql.exec.CommonJoinOperator.checkAndGenObject(CommonJoinOperator.java:756)
    at
org.apache.hadoop.hive.ql.exec.MapJoinOperator.process(MapJoinOperator.java:414)
    ... 13 more


Regards
Sanjiv Singh
Mob :  +091 9990-447-339

Mime
View raw message