hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rohith Sharma K S (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2185) Use pipes when localizing archives
Date Tue, 30 Jan 2018 07:22:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-2185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16344605#comment-16344605
] 

Rohith Sharma K S commented on YARN-2185:
-----------------------------------------

This is breaking in secured cluster with permission denied exception while localizing archives.
[~miklos.szegedi@cloudera.com] could you help to check this as it is blocking in secure cluster?

{noformat}
18/01/29 09:27:19 INFO mapreduce.Job:  map 0% reduce 0%
18/01/29 09:27:19 INFO mapreduce.Job: Job job_1517214437819_0004 failed with state FAILED
due to: Application application_1517214437819_0004 failed 20 times due to AM Container for
appattempt_1517214437819_0004_000020 exited with  exitCode: 1
Failing this attempt.Diagnostics: [2018-01-29 09:27:17.501]Exception from container-launch.
Container id: container_e20_1517214437819_0004_20_000001
Exit code: 1
Shell output: main : command provided 1
main : run as user is hrt_qa
main : requested yarn user is hrt_qa
Getting exit code file...
Creating script paths...
Writing pid file...
Writing to tmp file /grid/0/hadoop/yarn/local/nmPrivate/application_1517214437819_0004/container_e20_1517214437819_0004_20_000001/container_e20_1517214437819_0004_20_000001.pid.tmp
Writing to cgroup task files...
Creating local dirs...
Launching container...
Getting exit code file...
Creating script paths...


[2018-01-29 09:27:17.503]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
find: ?./mr-framework?: Permission denied

[2018-01-29 09:27:17.503]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
find: ?./mr-framework?: Permission denied

For more detailed output, check the application tracking page: http://ctr-e137-1514896590304-41842-01-000006.hwx.site:8088/cluster/app/application_1517214437819_0004
Then click on links to logs of each attempt.
. Failing the application.
18/01/29 09:27:19 INFO mapreduce.Job: Counters: 0
Job job_1517214437819_0004 failed!
{noformat}

> Use pipes when localizing archives
> ----------------------------------
>
>                 Key: YARN-2185
>                 URL: https://issues.apache.org/jira/browse/YARN-2185
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: nodemanager
>    Affects Versions: 2.4.0
>            Reporter: Jason Lowe
>            Assignee: Miklos Szegedi
>            Priority: Major
>             Fix For: 3.1.0
>
>         Attachments: YARN-2185.000.patch, YARN-2185.001.patch, YARN-2185.002.patch, YARN-2185.003.patch,
YARN-2185.004.patch, YARN-2185.005.patch, YARN-2185.006.patch, YARN-2185.007.patch, YARN-2185.008.patch,
YARN-2185.009.patch, YARN-2185.010.patch, YARN-2185.011.patch, YARN-2185.012.patch, YARN-2185.012.patch
>
>
> Currently the nodemanager downloads an archive to a local file, unpacks it, and then
removes it.  It would be more efficient to stream the data as it's being unpacked to avoid
both the extra disk space requirements and the additional disk activity from storing the archive.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message