hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Haibo Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5765) LinuxContainerExecutor creates appcache and its subdirectories with wrong group owner.
Date Mon, 07 Nov 2016 18:50:58 GMT

    [ https://issues.apache.org/jira/browse/YARN-5765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15645043#comment-15645043
] 

Haibo Chen commented on YARN-5765:
----------------------------------

Thanks [~Naganarasimha] for the patch! I'd expect this to work since I have tested a similar
change. While I am testing it in a cluster, we can discuss what is the best place to put the
umask() system call. IMHO, create_validate_dir may not be the best place to put it. The reason
being processes that execute mkdirs() may not notice their umasks have been modified to 0027
in create_validate_dir(). If a process, say root (I am not sure though whether umask(0027)
is  unwanted by root), wants to use a different umask other than 0027, it will need to remember
to call umask again after mkdirs().  Instead, I think we can call umask() right after fork
in launch_container_as_user() or in create_container_directories(). This way, only umask of
the child process that runs as the user will be changed to 0027. 

> LinuxContainerExecutor creates appcache and its subdirectories with wrong group owner.
> --------------------------------------------------------------------------------------
>
>                 Key: YARN-5765
>                 URL: https://issues.apache.org/jira/browse/YARN-5765
>             Project: Hadoop YARN
>          Issue Type: Bug
>    Affects Versions: 2.8.0, 3.0.0-alpha1
>            Reporter: Haibo Chen
>            Assignee: Naganarasimha G R
>            Priority: Blocker
>         Attachments: YARN-5765.001.patch
>
>
> LinuxContainerExecutor creates usercache/\{userId\}/appcache/\{appId\} with wrong group
owner, causing Log aggregation and ShuffleHandler to fail because node manager process does
not have permission to read the files under the directory.
> This can be easily reproduced by enabling LCE and submitting a MR example job as a user
that does not belong to the same group that NM process belongs to. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message