hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Miklos Szegedi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5765) LinuxContainerExecutor creates appcache and its subdirectories with wrong group owner.
Date Thu, 10 Nov 2016 22:30:58 GMT

    [ https://issues.apache.org/jira/browse/YARN-5765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15655375#comment-15655375
] 

Miklos Szegedi commented on YARN-5765:
--------------------------------------

Thank you, [~Naganarasimha] for the patch and [~haibochen] for the review. If I understand
it correctly, this is the flow of calls.
{code}
launch_container_as_user
  fork
    create_local_dirs
      create_log_dirs
        mkdir
      change_effective_user
      create_container_directories
        mkdirs
          create_validate_dir
{code}
We cannot change umask before change_effective_user() I think and changing it in mkdirs()
or create_validate_dir() may add side effects to other callers of mkdirs() in the future as
[~haibochen] mentioned. What I would do is to set the umask at the beginning of create_container_directories
right at the comment below
{code}
// create dirs as 0750
umask(0027);
{code}
I would also reset it to the previous value, before it returns.
Just a side note: This is what the Linux man page says about mkdir(): "in the absence of a
default ACL, the mode of the created directory is       (mode & ~umask & 0777)"
This means that by removing chmod this change does not apply to cases anymore, when the default
ACL is too restrictive. Could this be an issue, or do we rely on the admin to set the default
ACL correctly?

> LinuxContainerExecutor creates appcache and its subdirectories with wrong group owner.
> --------------------------------------------------------------------------------------
>
>                 Key: YARN-5765
>                 URL: https://issues.apache.org/jira/browse/YARN-5765
>             Project: Hadoop YARN
>          Issue Type: Bug
>    Affects Versions: 2.8.0, 3.0.0-alpha1
>            Reporter: Haibo Chen
>            Assignee: Naganarasimha G R
>            Priority: Blocker
>         Attachments: YARN-5765.001.patch
>
>
> LinuxContainerExecutor creates usercache/\{userId\}/appcache/\{appId\} with wrong group
owner, causing Log aggregation and ShuffleHandler to fail because node manager process does
not have permission to read the files under the directory.
> This can be easily reproduced by enabling LCE and submitting a MR example job as a user
that does not belong to the same group that NM process belongs to. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message