hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Templeton (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (MAPREDUCE-6704) Container fail to launch for mapred application
Date Thu, 27 Oct 2016 16:30:59 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15612421#comment-15612421
] 

Daniel Templeton edited comment on MAPREDUCE-6704 at 10/27/16 4:30 PM:
-----------------------------------------------------------------------

+1 to #3.  I agree with [~aw] that a little pollution is OK if it makes the out-of-box experience
significantly better.  And I don't think spilling over an environment variable is so bad anyway.

#1 has several issues.  In addition to what was already stated, the vars are evaluated on
the client, so if the client doesn't have the same paths as the NM hosts, it won't work.

#2 is a code-level pollution, which is a more serious thing than #3.

#4 doesn't sound like the right idea to me.  Users don't need to control the whitelisting
on a per-job basis, so it's solving a problem that doesn't exist.  It also has some security
implications that need to be carefully considered.


was (Author: templedf):
+1 to #3.  A agree with [~aw] that a little pollution is OK if it makes the out-of-box experience
significantly better.  And I don't think spilling over an environment variable is so bad anyway.

#1 has several issues.  In addition to what was already stated, the vars are evaluated on
the client, so if the client doesn't have the same paths as the NM hosts, it won't work.

#2 is a code-level pollution, which is a more serious thing than #3.

#4 doesn't sound like the right idea to me.  Users don't need to control the whitelisting
on a per-job basis, so it's solving a problem that doesn't exist.  It also has some security
implications that need to be carefully considered.

> Container fail to launch for mapred application
> -----------------------------------------------
>
>                 Key: MAPREDUCE-6704
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6704
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Bibin A Chundatt
>            Priority: Blocker
>         Attachments: 0001-MAPREDUCE-6704.patch, 0001-YARN-5026.patch, container-whitelist-env-wip.patch
>
>
> Container fail to launch for mapred application.
> As part for launch script {{HADOOP_MAPRED_HOME}} default value is not set .After https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c
  {{HADOOP_MAPRED_HOME}} is not able to get from {{builder.environment()}} since {{DefaultContainerExecutor#buildCommandExecutor}}
sets inherit to false.
> {noformat}
> 16/05/02 09:16:05 INFO mapreduce.Job: Job job_1462155939310_0004 failed with state FAILED
due to: Application application_1462155939310_0004 failed 2 times due to AM Container for
appattempt_1462155939310_0004_000002 exited with  exitCode: 1
> Failing this attempt.Diagnostics: Exception from container-launch.
> Container id: container_1462155939310_0004_02_000001
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
>         at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
>         at org.apache.hadoop.util.Shell.run(Shell.java:850)
>         at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
>         at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
>         at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
>         at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
>         at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier; support
was removed in 8.0
> Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier; support
was removed in 8.0
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org


Mime
View raw message