hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Templeton (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6704) Container fail to launch for mapred application
Date Fri, 28 Oct 2016 13:48:58 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15615461#comment-15615461
] 

Daniel Templeton commented on MAPREDUCE-6704:
---------------------------------------------

As if we're not split enough on this decision, let me throw in one more option.

The final and correct answer is that users should set up distributed cache deployment (per
[https://hadoop.apache.org/docs/r3.0.0-alpha1/hadoop-mapreduce-client/hadoop-mapreduce-client-core/DistributedCacheDeploy.html]).
 At that point things just work, and upgrades become simpler.  The problem is that distributed
cache deployment isn't an option for out of the box.

Spark does not have the problem we're trying to solve here.  Every time you submit a Spark
application, they ship the assembly JAR via the distributed cache.  If we want to push users
to distributed cache deployment, maybe the way to solve the out-of-box problem is the light
version of dist cache deployment: -libjars.

Option #5) Add a MapReduce property that controls whether the MapReduce JARs are automatically
shipped with the job via -libjars, and turn it on by default.  Yes, it's inefficient (in both
time and space), but it works out of the box and is a natural segue into dist cache deployment.

> Container fail to launch for mapred application
> -----------------------------------------------
>
>                 Key: MAPREDUCE-6704
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6704
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Bibin A Chundatt
>            Priority: Blocker
>         Attachments: 0001-MAPREDUCE-6704.patch, 0001-YARN-5026.patch, container-whitelist-env-wip.patch
>
>
> Container fail to launch for mapred application.
> As part for launch script {{HADOOP_MAPRED_HOME}} default value is not set .After https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c
  {{HADOOP_MAPRED_HOME}} is not able to get from {{builder.environment()}} since {{DefaultContainerExecutor#buildCommandExecutor}}
sets inherit to false.
> {noformat}
> 16/05/02 09:16:05 INFO mapreduce.Job: Job job_1462155939310_0004 failed with state FAILED
due to: Application application_1462155939310_0004 failed 2 times due to AM Container for
appattempt_1462155939310_0004_000002 exited with  exitCode: 1
> Failing this attempt.Diagnostics: Exception from container-launch.
> Container id: container_1462155939310_0004_02_000001
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
>         at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
>         at org.apache.hadoop.util.Shell.run(Shell.java:850)
>         at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
>         at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
>         at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
>         at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
>         at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier; support
was removed in 8.0
> Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier; support
was removed in 8.0
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org


Mime
View raw message