hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "brahim bahri (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-9187) Hive0.14.0 query fail with Tez0.5.3 and Hadoop2.4.1
Date Thu, 21 May 2015 13:13:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-9187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14554251#comment-14554251
] 

brahim bahri commented on HIVE-9187:
------------------------------------

i have the same problem, did you please found a solution

> Hive0.14.0 query fail with Tez0.5.3 and Hadoop2.4.1
> ---------------------------------------------------
>
>                 Key: HIVE-9187
>                 URL: https://issues.apache.org/jira/browse/HIVE-9187
>             Project: Hive
>          Issue Type: Bug
>         Environment: CentOS 6
>            Reporter: Ethan Hu
>            Priority: Critical
>
> The Operation System´╝îCentOS 6
> Software version, Hadoop 2.4.1, Tez 0.5.3, Hive 0.14.0.
> The hive table structure is 
> CREATE EXTERNAL TABLE IF NOT EXISTS inv_info(optDate string, code int, custno int, sku
int, euno int, totalamount double, price double, type int, finalamout double) ROW FORMAT DELIMITED
FIELDS TERMINATED BY ',' LOCATION '/user/hadoop/inv';
> Use Hive local Cli for query,
> 1. Use the default engine "mr", execute the Hive QL, it's normal
> hive> select count(*) from inv_info;
> Automatically selecting local only mode for query
> Query ID = root_20141221181313_60e150e1-34dd-4bea-8d4f-1a703c5516d6
> Total jobs = 1
> Launching Job 1 out of 1
> Number of reduce tasks determined at compile time: 1
> In order to change the average load for a reducer (in bytes):
>   set hive.exec.reducers.bytes.per.reducer=<number>
> In order to limit the maximum number of reducers:
>   set hive.exec.reducers.max=<number>
> In order to set a constant number of reducers:
>   set mapreduce.job.reduces=<number>
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in [jar:file:/webapp/hadoop-2.4.1/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in [jar:file:/webapp/apache-hive-0.14.0-bin/lib/hive-jdbc-0.14.0-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> 14/12/21 18:13:50 WARN util.NativeCodeLoader: Unable to load native-hadoop library for
your platform... using builtin-java classes where applicable
> 14/12/21 18:13:51 WARN conf.Configuration: file:/tmp/root/2006970b-1b5e-41e7-a604-4ed7405cc7e9/hive_2014-12-21_18-13-41_747_2372484449278617092-3/-local-10003/jobconf.xml:an
attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
> 14/12/21 18:13:51 WARN conf.Configuration: file:/tmp/root/2006970b-1b5e-41e7-a604-4ed7405cc7e9/hive_2014-12-21_18-13-41_747_2372484449278617092-3/-local-10003/jobconf.xml:an
attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
> Execution log at: /tmp/root/root_20141221181313_60e150e1-34dd-4bea-8d4f-1a703c5516d6.log
> Job running in-process (local Hadoop)
> Hadoop job information for null: number of mappers: 0; number of reducers: 0
> 2014-12-21 18:14:00,666 null map = 0%,  reduce = 0%
> 2014-12-21 18:14:05,365 null map = 100%,  reduce = 100%
> Ended Job = job_local886866076_0001
> Execution completed successfully
> MapredLocal task succeeded
> OK
> 2146872
> Time taken: 26.281 seconds, Fetched: 1 row(s)
> 2. when I  set hive.execution.engine=tez; the query is fail, can anyone help me to find
the reason? I looked up for many reference materials, can't resolve it.
> hive> select count(*) from inv_info;
> Query ID = root_20141221181515_cbf8f20b-e105-441e-b044-269266088c48
> Total jobs = 1
> Launching Job 1 out of 1
> Status: Running (Executing on YARN cluster with App id application_1418953370736_0009)
> --------------------------------------------------------------------------------
>         VERTICES      STATUS  TOTAL  COMPLETED  RUNNING  PENDING  FAILED  KILLED
> --------------------------------------------------------------------------------
> Map 1                 FAILED      4          0        0        4       9       3
> Reducer 2             KILLED      1          0        0        1       0       1
> --------------------------------------------------------------------------------
> VERTICES: 00/02  [>>--------------------------] 0%    ELAPSED TIME: 29.57 s
> --------------------------------------------------------------------------------
> Status: Failed
> Vertex failed, vertexName=Map 1, vertexId=vertex_1418953370736_0009_1_00, diagnostics=[Task
failed, taskId=task_1418953370736_0009_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Container
container_1418953370736_0009_01_000002 finished with diagnostics set to [Container failed.
Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException:
> org.apache.hadoop.util.Shell$ExitCodeException:
>         at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
>         at org.apache.hadoop.util.Shell.run(Shell.java:418)
>         at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
>         at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
>         at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300)
>         at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:701)
> Container exited with a non-zero exit code 1
> ]], TaskAttempt 1 failed, info=[Container container_1418953370736_0009_01_000009 finished
with diagnostics set to [Container failed. Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException:
> org.apache.hadoop.util.Shell$ExitCodeException:
>         at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
>         at org.apache.hadoop.util.Shell.run(Shell.java:418)
>         at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
>         at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
>         at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300)
>         at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:701)
> Container exited with a non-zero exit code 1
> ]], TaskAttempt 2 failed, info=[Container container_1418953370736_0009_01_000010 finished
with diagnostics set to [Container failed. Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException:
> org.apache.hadoop.util.Shell$ExitCodeException:
>         at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
>         at org.apache.hadoop.util.Shell.run(Shell.java:418)
>         at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
>         at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
>         at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300)
>         at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:701)
> Container exited with a non-zero exit code 1
> ]], TaskAttempt 3 failed, info=[Container container_1418953370736_0009_01_000013 finished
with diagnostics set to [Container failed. Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException:
> org.apache.hadoop.util.Shell$ExitCodeException:
>         at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
>         at org.apache.hadoop.util.Shell.run(Shell.java:418)
>         at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
>         at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
>         at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300)
>         at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:701)
> Container exited with a non-zero exit code 1
> ]]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1418953370736_0009_1_00
[Map 1] killed/failed due to:null]
> Vertex killed, vertexName=Reducer 2, vertexId=vertex_1418953370736_0009_1_01, diagnostics=[Vertex
received Kill while in RUNNING state., Vertex killed as other vertex failed. failedTasks:0,
Vertex vertex_1418953370736_0009_1_01 [Reducer 2] killed/failed due to:null]
> DAG failed due to vertex failure. failedVertices:1 killedVertices:1
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message