hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Íñigo Goiri (JIRA) <j...@apache.org>
Subject [jira] [Comment Edited] (HADOOP-15528) Deprecate ContainerLaunch#link by using FileUtil#SymLink
Date Fri, 06 Jul 2018 20:30:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-15528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535317#comment-16535317
] 

Íñigo Goiri edited comment on HADOOP-15528 at 7/6/18 8:29 PM:
--------------------------------------------------------------

Thanks [~yqwang] for the comments.

Totally agree with you overall.
 # We need to add security checks. We can implement the security check in a way that we are
way more secure than the old behavior. As [~stevel@apache.org] said we should improve security
to get more approval from OSS community.

{quote}
The old behavior is the symlink operation is executed in the batch script, which is executed
as a child process under some limited privileged and resource isolation environment, such
as windows job object (with windows secure container) or linux cgroups, etc. 
However, the new behavior is the symlink operation is executed by NM itself, which is executed
as a child process under NM itself, it shares the same execution environment as NM.
{quote}

In the old behavior, even if we run symlink with limited privileges we still do not check
the content of CLC.
In the new implementation, I am planning to try to restrict the privileges and add checks
on the content of CLC.
 # As I said before, in case of error we should avoid starting the container. I would like
to still keep the old behavior, aka start the container and exit with a better error log.
Let me try to improve the exit message.
 # The old behavior does not allow to retry. We can add a retry logic in a future Jira as
an improvement.

I don't know if we should work for the security aspects in this Jira or in next jira(s).


was (Author: giovanni.fumarola):
Thanks [~yqwang] for the comments.

Totally agree with you overall.
 # We need to add security checks. We can implement the security check in a way that we are
way more secure than the old behavior. As [~stevel@apache.org] said we should improve security
to get more approval from OSS community.

{noformat}
The old behavior is the symlink operation is executed in the batch script, which is executed
as a child process under some limited privileged and resource isolation environment, such
as windows job object (with windows secure container) or linux cgroups, etc. 
However, the new behavior is the symlink operation is executed by NM itself, which is executed
as a child process under NM itself, it shares the same execution environment as NM.{noformat}
In the old behavior, even if we run symlink with limited privileges we still do not check
the content of CLC.
In the new implementation, I am planning to try to restrict the privileges and add checks
on the content of CLC.
 # As I said before, in case of error we should avoid starting the container. I would like
to still keep the old behavior, aka start the container and exit with a better error log.
Let me try to improve the exit message.
 # The old behavior does not allow to retry. We can add a retry logic in a future Jira as
an improvement.

I don't know if we should work for the security aspects in this Jira or in next jira(s).

> Deprecate ContainerLaunch#link by using FileUtil#SymLink
> --------------------------------------------------------
>
>                 Key: HADOOP-15528
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15528
>             Project: Hadoop Common
>          Issue Type: Sub-task
>            Reporter: Giovanni Matteo Fumarola
>            Assignee: Giovanni Matteo Fumarola
>            Priority: Major
>         Attachments: HADOOP-15528-HADOOP-15461.v1.patch, HADOOP-15528-HADOOP-15461.v2.patch,
HADOOP-15528-HADOOP-15461.v3.patch
>
>
> {{ContainerLaunch}} currently uses its own utility to create links (including winutils).
> This should be deprecated and rely on {{FileUtil#SymLink}} which is already multi-platform
and pure Java.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message