hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matt McCline (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-11051) Hive 1.2.0 MapJoin w/Tez - LazyBinaryArray cannot be cast to [Ljava.lang.Object;
Date Wed, 24 Jun 2015 23:00:14 GMT

    [ https://issues.apache.org/jira/browse/HIVE-11051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600325#comment-14600325
] 

Matt McCline commented on HIVE-11051:
-------------------------------------

[~wzheng] Can you give this a non-binding +1?  Thanks

> Hive 1.2.0  MapJoin w/Tez - LazyBinaryArray cannot be cast to [Ljava.lang.Object;
> ---------------------------------------------------------------------------------
>
>                 Key: HIVE-11051
>                 URL: https://issues.apache.org/jira/browse/HIVE-11051
>             Project: Hive
>          Issue Type: Bug
>          Components: Serializers/Deserializers, Tez
>    Affects Versions: 1.2.0
>            Reporter: Greg Senia
>            Assignee: Matt McCline
>            Priority: Critical
>         Attachments: HIVE-11051.01.patch, problem_table_joins.tar.gz
>
>
> I tried to apply: HIVE-10729 which did not solve the issue.
> The following exception is thrown on a Tez MapJoin with Hive 1.2.0 and Tez 0.5.4/0.5.3
> {code}
> Status: Running (Executing on YARN cluster with App id application_1434641270368_1038)
> --------------------------------------------------------------------------------
>         VERTICES      STATUS  TOTAL  COMPLETED  RUNNING  PENDING  FAILED  KILLED
> --------------------------------------------------------------------------------
> Map 1 ..........   SUCCEEDED      3          3        0        0       0       0
> Map 2 ...             FAILED      3          1        0        2       7       0
> --------------------------------------------------------------------------------
> VERTICES: 01/02  [=================>>---------] 66%   ELAPSED TIME: 7.39 s    

> --------------------------------------------------------------------------------
> Status: Failed
> Vertex failed, vertexName=Map 2, vertexId=vertex_1434641270368_1038_2_01, diagnostics=[Task
failed, taskId=task_1434641270368_1038_2_01_000002, diagnostics=[TaskAttempt 0 failed, info=[Error:
Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
Hive Runtime Error while processing row {"cnctevn_id":"002245282386","svcrqst_id":"0000003627217285","svcrqst_crt_dts":"2015-04-23
11:54:39.238357","subject_seq_no":1,"plan_component":"HMOM1     ","cust_segment":"RM ","cnctyp_cd":"001","cnctmd_cd":"D02","cnctevs_cd":"007","svcrtyp_cd":"335","svrstyp_cd":"088","cmpltyp_cd":"
","catsrsn_cd":"        ","apealvl_cd":" ","cnstnty_cd":"001","svcrqst_asrqst_ind":"Y","svcrqst_rtnorig_in":"N","svcrqst_vwasof_dt":"null","sum_reason_cd":"98","sum_reason":"Exclude","crsr_master_claim_index":null,"svcrqct_cds":["
  "],"svcrqst_lupdt":"2015-04-23 22:14:01.288132","crsr_lupdt":null,"cntevsds_lupdt":"2015-04-23
11:54:40.740061","ignore_me":1,"notes":null}
>         at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:171)
>         at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:137)
>         at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:324)
>         at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
>         at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
>         at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
>         at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:744)
> Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
Hive Runtime Error while processing row {"cnctevn_id":"002245282386","svcrqst_id":"0000003627217285","svcrqst_crt_dts":"2015-04-23
11:54:39.238357","subject_seq_no":1,"plan_component":"HMOM1     ","cust_segment":"RM ","cnctyp_cd":"001","cnctmd_cd":"D02","cnctevs_cd":"007","svcrtyp_cd":"335","svrstyp_cd":"088","cmpltyp_cd":"
","catsrsn_cd":"        ","apealvl_cd":" ","cnstnty_cd":"001","svcrqst_asrqst_ind":"Y","svcrqst_rtnorig_in":"N","svcrqst_vwasof_dt":"null","sum_reason_cd":"98","sum_reason":"Exclude","crsr_master_claim_index":null,"svcrqct_cds":["
  "],"svcrqst_lupdt":"2015-04-23 22:14:01.288132","crsr_lupdt":null,"cntevsds_lupdt":"2015-04-23
11:54:40.740061","ignore_me":1,"notes":null}
>         at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:91)
>         at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:68)
>         at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:290)
>         at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:148)
>         ... 13 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row {"cnctevn_id":"002245282386","svcrqst_id":"0000003627217285","svcrqst_crt_dts":"2015-04-23
11:54:39.238357","subject_seq_no":1,"plan_component":"HMOM1     ","cust_segment":"RM ","cnctyp_cd":"001","cnctmd_cd":"D02","cnctevs_cd":"007","svcrtyp_cd":"335","svrstyp_cd":"088","cmpltyp_cd":"
","catsrsn_cd":"        ","apealvl_cd":" ","cnstnty_cd":"001","svcrqst_asrqst_ind":"Y","svcrqst_rtnorig_in":"N","svcrqst_vwasof_dt":"null","sum_reason_cd":"98","sum_reason":"Exclude","crsr_master_claim_index":null,"svcrqct_cds":["
  "],"svcrqst_lupdt":"2015-04-23 22:14:01.288132","crsr_lupdt":null,"cntevsds_lupdt":"2015-04-23
11:54:40.740061","ignore_me":1,"notes":null}
>         at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:518)
>         at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:83)
>         ... 16 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unexpected exception: org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryArray
cannot be cast to [Ljava.lang.Object;
>         at org.apache.hadoop.hive.ql.exec.MapJoinOperator.process(MapJoinOperator.java:426)
>         at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:837)
>         at org.apache.hadoop.hive.ql.exec.FilterOperator.process(FilterOperator.java:122)
>         at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:837)
>         at org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:97)
>         at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:162)
>         at org.apache.hadoop.hive.ql.exec.Map
> {code}
> To Recreate the issue:
> {code}
> import table service_request_clean from  '/user/user/service_request_clean';
> import table ct_events_clean from  '/user/user/ct_events_clean';
> drop table ct_events1_test;
> create table ct_events1_test
> as select  a.*,
> b.svcrqst_id,
> b.svcrqct_cds,
> b.svcrtyp_cd,
> b.cmpltyp_cd,
> b.sum_reason_cd as src,
> b.cnctmd_cd,
> b.notes
> from ct_events_clean a
> inner join
> service_request_clean b
> on a.contact_event_id = b.cnctevn_id;
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message