hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yin Huai <huaiyin....@gmail.com>
Subject Re: NPE org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTable
Date Fri, 11 Oct 2013 15:12:04 GMT
Hello Xinyang,

Can you attach the query plan (the output of "EXPLAIN")? I think a bad plan
caused the error.

Also, can you try hive trunk? Looks like it is a bug fixed after the
release of 0.11.

Thanks,

Yin


On Fri, Oct 11, 2013 at 9:21 AM, xinyan Yang <moon.yang85@gmail.com> wrote:

> Development environment,hive 0.11、hadoop 1.0.3
>
>
> 2013/10/11 xinyan Yang <moon.yang85@gmail.com>
>
>> Hi,
>> when i run this sql,it fails,can anyone give me a advise
>>
>> --------------------------------------------------------------------------------------------------------------------------------
>> select e.udid as udid,e.app_id as app_id
>> from acorn_3g.ClientChannelDefine cc
>> join (
>> select udid,app_id,from_id
>> from (
>>  select u.device_id as udid,u.app_id as app_id,g.device_id as
>> 3gdid,u.from_id as from_id from acorn_3g.user_device_info u
>> left outer join (select device_id from acorn_3g.3g_device_id where
>> log_date<'2013-09-15') g
>>  on u.device_id=g.device_id where u.log_date='2013-09-15' and
>> u.from_id<>0 and u.type=1) f1
>> where 3gdid is null ) e
>> on(e.from_id=cc.from_id)
>>
>> --------------------------------------------------------------------------------------------------------------------------------
>>
>> error info:
>> Task with the most failures(4):
>> -----
>> Task ID:
>>   task_201305281414_236693_m_000001
>>
>> URL:
>>
>> http://YZSJHL18-22.opi.com:50030/taskdetails.jsp?jobid=job_201305281414_236693&tipid=task_201305281414_236693_m_000001<http://yzsjhl18-22.opi.com:50030/taskdetails.jsp?jobid=job_201305281414_236693&tipid=task_201305281414_236693_m_000001>
>> -----
>> Diagnostic Messages for this Task:
>> java.lang.RuntimeException:
>> org.apache.hadoop.hive.ql.metadata.HiveException:
>> java.lang.NullPointerException
>>         at
>> org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:162)
>>         at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
>>         at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436)
>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
>>         at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
>>         at java.security.AccessController.doPrivileged(Native Method)
>>         at javax.security.auth.Subject.doAs(Subject.java:396)
>>         at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
>>         at org.apache.hadoop.mapred.Child.main(Child.java:249)
>> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
>> java.lang.NullPointerException
>>         at
>> org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTable(MapJoinOperator.java:198)
>>         at
>> org.apache.hadoop.hive.ql.exec.MapJoinOperator.cleanUpInputFileChangedOp(MapJoinOperator.java:212)
>>         at
>> org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1377)
>>         at
>> org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1381)
>>         at
>> org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1381)
>>         at
>> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:611)
>>         at
>> org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:144)
>>         ... 8 more
>> Caused by: java.lang.NullPointerException
>>         at
>> org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTable(MapJoinOperator.java:186)
>>         ... 14 more
>>
>>
>> FAILED: Execution Error, return code 2 from
>> org.apache.hadoop.hive.ql.exec.MapRedTask
>> MapReduce Jobs Launched:
>> Job 0: Map: 343  Reduce: 2   Cumulative CPU: 3478.61 sec   HDFS Read:
>> 1862106687 HDFS Write: 3838425 SUCCESS
>> Job 1: Map: 2   HDFS Read: 0 HDFS Write: 0 FAIL
>>
>>
>

Mime
View raw message