hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1972) Implement secure Windows Container Executor
Date Wed, 28 May 2014 20:09:02 GMT

    [ https://issues.apache.org/jira/browse/YARN-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14011515#comment-14011515
] 

Vinod Kumar Vavilapalli commented on YARN-1972:
-----------------------------------------------

Thanks for working on this Remus. Can you upload the "short design"?

Questions/comments on the approach and the patch in the mean-while

h4. Approach
 - What are the requirements on the NodeManager user? Can it run as a regular 'yarn' user,
spawn the winutils shell and automatically launch task as some other user? Is there any admin
setup that is needed for this to say grant such privileges to 'yarn' user?
 - One reason why we resorted to duplicate most of the code in DefaultContainerExecutor in
container-executor.c for linux is performance. You are launching so many commands for every
container - to chown files, to copy files etc. You should measure the performance impact of
this to figure out if what the patch does is fine or if we should imitate what the linux-executor
does.

h4. Patch
WindowsSecureContainerExecutor
 - The overridden getRunCommand skips things like the setting niceness feature (YARN-443)
in linux. Arguably this isn't working in non-secure mode before anyways. Is there a way we
can bump process-priority in windows? If so, when we add that feature, we'll need to be careful
to change both the default and the secure Executor.
 - namenodeGroup -> nodeManagerGroup
 -  The division of responsibility between launching multiple commands before starting the
localizer and the stuff that happens inside the localizer: Localizer already does createUserLocalDirs
etc. So you don't need to do them explicitly in the java code inside NodeManager process.
 - In the minimum we should definitely move exec.localizeClasspathJar() related stuff into
the winutils start-process code.
 - Why is appLocalizationCounter needed? Once we tackle container-preserving NM-restart (YARN-1336),
this will be an issue. Why cannot we simply use the localizerId? That is unique enough if
we want uniqueness.
 - Also the startLocalizer() method is a near clone of what exists in LinuxContainerExecutor.
We should refactor and reuse, otherwise it will be a maintenance headache.

> Implement secure Windows Container Executor
> -------------------------------------------
>
>                 Key: YARN-1972
>                 URL: https://issues.apache.org/jira/browse/YARN-1972
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: nodemanager
>            Reporter: Remus Rusanu
>            Assignee: Remus Rusanu
>              Labels: security, windows
>         Attachments: YARN-1972.1.patch
>
>
> This work item represents the Java side changes required to implement a secure windows
container executor, based on the YARN-1063 changes on native/winutils side. 
> Necessary changes include leveraging the winutils task createas to launch the container
process as the required user and a secure localizer (launch localization as a separate process
running as the container user).



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message