hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jihyun Suh <jhsuh.ourli...@gmail.com>
Subject # of failed Map Tasks exceeded allowed limit
Date Fri, 05 Dec 2014 00:09:54 GMT
I have a problem that inserting data from external table to managed table.

I already have done just 'limit 10' with below hql but can't all data.

I have the error just like '# of failed Map Tasks exceeded allowed limit'

How should I do?

---------------------------------------------------------------------------------

cause: SQLException:Error while executing statement:



INSERT OVERWRITE TABLE newTbl PARTITION(etl_ymd, etl_hh)

SELECT rec_type,

    hash(substr(svd_name,1,case when instr(svd_name,'F')=0 then
length(svd_name) when instr(svd_name,'F')>30 then 30 else
instr(svd_name,'F')-1 end )) as imsi,

    hash(substr(svd_nick,1,case when instr(svd_nick,'F')=0 then
length(svd_nick) when instr(svd_nick,'F')>0 then 30 else
instr(svd_nick,'F')-1 end)) as imei,

    hash(a.private) as private,

    hash(case when substr(a.private,3,2)='82' then
concat('0',substr(a.private,5)) else substr(a.private,5) end) as num,

    evt_timestamp,

    msg_ref,

    hash(dest_no) as
dest_no,

        system_type,

    a.file_name as
file_name,


        concat(substr(a.file_name,2,2),substr(a.file_name,6,1))  as own_no,


etl_ymd,


        etl_hh

FROM
(

        select *,

        substr(svd_private,1,case when instr(svd_private,'F')=0 then
length(svd_private) when instr(svd_private,'F')>30 then 30 else
instr(svd_private,'F')-1 end) as private,

        split(split(INPUT__FILE__NAME, "etl_hh=[0-9]+/")[1], ".DAT")[0] as
file_name

from oldTbl

) a



by FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.MapRedTask, SQLState = 08S01, ResponseCode =
2

org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:168)

org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:156)

org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:198)

com.nexr.ndap.driver.hive.jdbc.HiveJdbcConnector.doExecute(HiveJdbcConnector.java:169)

com.nexr.ndap.driver.hive.jdbc.HiveJdbcConnector.executeQueryWithSession(HiveJdbcConnector.java:856)

com.nexr.ndap.driver.hive.ConcurrentHiveConnector.executeQueryWithSession(ConcurrentHiveConnector.java:283)

com.nexr.ndap.service.workbench.impl.HiveQueryServiceImpl.runQuery(HiveQueryServiceImpl.java:582)

com.nexr.ndap.service.workbench.impl.HiveQueryServiceImpl.runQuery(HiveQueryServiceImpl.java:433)

sun.reflect.GeneratedMethodAccessor743.invoke(Unknown Source)

sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

java.lang.reflect.Method.invoke(Method.java:597)

org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)

org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)

org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)

org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:110)

org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)

org.springframework.aop.interceptor.AsyncExecutionInterceptor$1.call(AsyncExecutionInterceptor.java:80)

java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)

java.util.concurrent.FutureTask.run(FutureTask.java:138)

java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)

java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)

java.lang.Thread.run(Thread.java:662)



--------------------------------------------------------------------------------

= job detail log

Hadoop job_201406192109_16353 on p-bigai-pk1-a03

User: admin

Job Name: INSERT OVERWRITE TABLE mzn.mzn_cdr_3gsms...a(Stage-1)

Job File:
hdfs://p-bigai-pk1-a03:9000/tmp/hadoop-mapred/mapred/staging/admin/.staging/job_201406192109_16353/job.xml

Submit Host: p-bigai-pk1-a01

Submit Host Address: 10.220.78.91

Job-ACLs: All users are allowed

Job Setup: Successful

Status: Failed

Failure Info:# of failed Map Tasks exceeded allowed limit. FailedCount: 1.
LastFailedTask: task_201406192109_16353_m_000006

Started at: Wed Dec 03 14:18:36 KST 2014

Failed at: Wed Dec 03 14:19:04 KST 2014

Failed in: 27sec

Job Cleanup: Successful


Black-listed TaskTrackers: 19

Mime
View raw message