hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Greg Phillips (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5280) Allow YARN containers to run with Java Security Manager
Date Tue, 10 Jan 2017 16:30:58 GMT

    [ https://issues.apache.org/jira/browse/YARN-5280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15815429#comment-15815429
] 

Greg Phillips commented on YARN-5280:
-------------------------------------

[~vvasudev] - I encountered some issues when attempting to move the generated java.policy
files into the container or application directories due to permissions conflicts when running
in secure mode.  Namely there are no container or application specific directories which allow
write access for the yarn user, and read access to the container run-as user in all configurations.
 This is resolved using the hadoop.tmp.dir following the example set by the DockerRuntime.
 The risk of running out of space on hadoop.tmp.dir should be small due to the following:
# Generated policy files are ~4KB, the largest yarn nodes can handle around 500 containers
making the hypothetical upper bound ~2MB of tmp usage.
# Policy files are deleted at the completion of container launch regardless of exit value,
as well as on nodemanager restart.  This functionality has been moved from reapContainer to
the end of launchContainer.

bq. Once we have the runtime support in, we can add support in MR and distributed shell for
the feature.

This patch has been tested extensively with MR to ensure all components (distributed cache,
libjars, etc.) work as intended.  The distributed shell will work if the distributed shell
jar is available under the hadoop home directory since all libraries in the hadoop home directory
are granted all permissions.  Cluster administrators will likely want to limit access to the
distributed shell jar to harden the cluster.

Please let me know if these compromises seem appropriate, or if there are additional steps
required to make this feature viable.

> Allow YARN containers to run with Java Security Manager
> -------------------------------------------------------
>
>                 Key: YARN-5280
>                 URL: https://issues.apache.org/jira/browse/YARN-5280
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: nodemanager, yarn
>    Affects Versions: 2.6.4
>            Reporter: Greg Phillips
>            Assignee: Greg Phillips
>            Priority: Minor
>              Labels: oct16-medium
>         Attachments: YARN-5280.001.patch, YARN-5280.002.patch, YARN-5280.003.patch, YARN-5280.004.patch,
YARN-5280.005.patch, YARN-5280.006.patch, YARN-5280.patch, YARNContainerSandbox.pdf
>
>
> YARN applications have the ability to perform privileged actions which have the potential
to add instability into the cluster. The Java Security Manager can be used to prevent users
from running privileged actions while still allowing their core data processing use cases.

> Introduce a YARN flag which will allow a Hadoop administrator to enable the Java Security
Manager for user code, while still providing complete permissions to core Hadoop libraries.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message