hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Badger (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4266) Allow whitelisted users to disable user re-mapping/squashing when launching docker containers
Date Thu, 06 Jul 2017 15:35:06 GMT

    [ https://issues.apache.org/jira/browse/YARN-4266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16076756#comment-16076756

Eric Badger commented on YARN-4266:

[~shanekumpf@gmail.com], sorry for the delay. I've been sidetracked a little bit. The patch
that I have still has a few bugs, but I'd like to make sure that we agree on the approach
before I move forward with cleaning it up and posting it here. 
bq. The intent to YARN-5534 is provide a mount white list, so I believe that should help here.
The initial patch here could hard code the bind mount while we test and provide feedback.
Hopefully we can leverage YARN-5534 before this is wrapped up.
Commented over on that jira. It would be good if we could get some traction over there as
it looks like that patch is pretty close to being done.

bq. I don't think this is a requirement for the initial version. We could do a follow on effort
to remove/reduce the need for the bind mounted socket for a known list of AMs, assuming the
behavior can be changed in those AMs.
This is true. I'm attempting to do my due diligence up front to see if there is an avenue
to get MRAppMaster to work without mounting /var/run/nscd. I've been talking with [~daryn]
offline who has done lots of work on UGI stuff and we're looking into solutions. One solution
that he suggested was going to back to our original idea of doing the adduser/usermod hack
during container startup. I don't like this as much as it only allows you to put the one user
in the container and will fail any other user lookups. It also would never get user/group
updates which might be relevant for long-running containers. And on top of that, it would
be unnecessary in the face of bind-mounting /var/run/nscd. However, it does get over the initial
obstacle of not being able to run without bind-mounting /var/run/nscd. Preferably, we can
find a way to make the --user switch work with MRAppMaster, but if not maybe this is the way
to go. Thoughts?

> Allow whitelisted users to disable user re-mapping/squashing when launching docker containers
> ---------------------------------------------------------------------------------------------
>                 Key: YARN-4266
>                 URL: https://issues.apache.org/jira/browse/YARN-4266
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: yarn
>            Reporter: Sidharta Seethana
>            Assignee: luhuichun
>         Attachments: YARN-4266.001.patch, YARN-4266.001.patch, YARN-4266_Allow_whitelisted_users_to_disable_user_re-mapping.pdf,
YARN-4266_Allow_whitelisted_users_to_disable_user_re-mapping_v2.pdf, YARN-4266_Allow_whitelisted_users_to_disable_user_re-mapping_v3.pdf,
> Docker provides a mechanism (the --user switch) that enables us to specify the user the
container processes should run as. We use this mechanism today when launching docker containers
. In non-secure mode, we run the docker container based on `yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user`
and in secure mode, as the submitting user. However, this mechanism breaks down with a large
number of 'pre-created' images which don't necessarily have the users available within the
image. Examples of such images include shared images that need to be used by multiple users.
We need a way in which we can allow a pre-defined set of users to run containers based on
existing images, without using the --user switch. There are some implications of disabling
this user squashing that we'll need to work through : log aggregation, artifact deletion etc.,

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message