hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rohith Sharma K S <rohithsharm...@huawei.com>
Subject RE: MR job fails with too many mappers
Date Tue, 18 Nov 2014 15:43:32 GMT
Hi

Could you get syserr and sysout log for contrainer.? These logs will be available in the same
location  syslog for container.
${yarn.nodemanager.log-dirs}/<app-id>/<container-id>
This helps to find problem!!


Thanks & Regards
Rohith Sharma K S

From: francexo83 [mailto:francexo83@gmail.com]
Sent: 18 November 2014 20:53
To: user@hadoop.apache.org
Subject: MR job fails with too many mappers

Hi All,

I have a small  hadoop cluster with three nodes and HBase 0.98.1 installed on it.

The hadoop version is 2.3.0 and below my use case scenario.

I wrote a map reduce program that reads data from an hbase table and does some transformations
on these data.
Jobs are very simple so they didn't need the  reduce phase. I also wrote a TableInputFormat
 extension in order to maximize the number of concurrent maps on the cluster.
In other words, each  row should be processed by a single map task.

Everything goes well until the number of rows and consequently  mappers exceeds 300000 quota.

This is the only exception I see when the job fails:

Application application_1416304409718_0032 failed 2 times due to AM Container for appattempt_1416304409718_0032_000002
exited with exitCode: 1 due to:


Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException:
org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:511)
at org.apache.hadoop.util.Shell.run(Shell.java:424)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:656)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Container exited with a non-zero exit code 1


Cluster configuration details:
Node1: 12 GB, 4 core
Node2: 6 GB, 4 core
Node3: 6 GB, 4 core

yarn.scheduler.minimum-allocation-mb=2048
yarn.scheduler.maximum-allocation-mb=4096
yarn.nodemanager.resource.memory-mb=6144



Regards
Mime
View raw message