hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hitesh Shah (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1020) Resource Localization using Groups as a new Localization Type
Date Fri, 02 Aug 2013 20:41:48 GMT

    [ https://issues.apache.org/jira/browse/YARN-1020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13728074#comment-13728074

Hitesh Shah commented on YARN-1020:

[~ojoshi] Could you explain how a container can modify a local resource file in the NM's private
directory? Shouldn't the container have only read/execute access on the local resource and
should never have write access to it? Do we really need an extra explicit group type? If the
remote file on HDFS has 750 permissions i.e. group readable but not world readable - would
this suffice?

> Resource Localization using Groups as a new Localization Type
> -------------------------------------------------------------
>                 Key: YARN-1020
>                 URL: https://issues.apache.org/jira/browse/YARN-1020
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Omkar Vinit Joshi
> The scenario is as follows..
> * We definitely will have multiple applications running on top of yarn. These applications
whenever run by users will need resources to be localized. Now the options what application-users
will have for localizing resources are:-
> ** APPLICATION ... these files will be available for only that instance of the application
and only for that single user. If we talk in terms of MR then for single job.
> ** PRIVATE ... available only for that user only for multiple runs of that application.
Other users clearly will not be able to take advantage of that. So ideally will be wasting
space (local resource cache) by replicating the same file again and again.
> ** PUBLIC... there will be only one copy of individual files of the application say APP_1..GOOD
..in the sense it will be accessible to all the users...But for secured clusters; users of
different application (say APP_2) containers can then gain easy access to this applications
(APP_1) private files and potentially may modify it.
> So clearly we don't have any solution today to solve the above problem with existing
RESOURCE_LOCALIZATION_TYPES without effectively using space. Therefore we need something like
GROUP to address this scenario.
> Thoughts?? 

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message