hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Oleksandr Shevchenko (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-8812) Containers fail during creating a symlink which started with hyphen for a resource file
Date Mon, 19 Nov 2018 16:00:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-8812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16691904#comment-16691904

Oleksandr Shevchenko commented on YARN-8812:

Could someone kindly review the patch?

> Containers fail during creating a symlink which started with hyphen for a resource file
> ---------------------------------------------------------------------------------------
>                 Key: YARN-8812
>                 URL: https://issues.apache.org/jira/browse/YARN-8812
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Oleksandr Shevchenko
>            Assignee: Oleksandr Shevchenko
>            Priority: Minor
>         Attachments: YARN-8812.001.patch
> When we run some job and add a file with alias started with hyphen then a container
fails during creating a symlink for a resource file:
> {noformat}
> yarn jar hadoop-mapreduce-examples.jar pi -files testfile#-symlink  1 1
> {noformat}
> or add a file to distributed cache in MR job by "job.addCacheFile"
> Containers fail if resource file has a symlink started with hyphen with the following
> {noformat}
> Stack trace: ExitCodeException exitCode=1: /tmp/hadoop-yarn/nm-local-dir/usercache/yarn/appcache/application_1537449069809_0022/container_e01_1537449069809_0022_02_000001/launch_container.sh
> ln: invalid option -- 'y'
> Try 'ln --help' for more information.
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:572)
> at org.apache.hadoop.util.Shell.run(Shell.java:466)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:768)
> at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:306)
> {noformat}
> The main cause of the problem is "launch_container.sh" script whitch contain the following
command for creating a symlink:
> {noformat}
> ln -sf "/tmp/hadoop-yarn/nm-local-dir/usercache/yarn/filecache/49/testfile" "-symlink"
> {noformat}
> As the result "-symlink" parse as "-s" flag but not as a symlink name.
> The same job successfully passed when running on MRv1 but not on YARN since symlinks
create in different ways. Unix systems support names which start with the hyphen.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message