hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hemanth Yamijala (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-856) Localized files from DistributedCache should have right access-control
Date Mon, 24 Aug 2009 06:54:59 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12746749#action_12746749
] 

Hemanth Yamijala commented on MAPREDUCE-856:
--------------------------------------------

Looked at the patch. I have a few comments:

- Make Localizer an instance class, as in general, that's a more flexible design, and also
there's state that the localizer is needing to maintain anyway.
- I would recommend initializeUserDirs to pass the taskcontroller instead of tasktracker,
as the entire tasktracker interface is not needed by the localizer atleast now.
- In HADOOP-4491, if the user directory cannot be created on any disk, we were failing localization.
I think that's a useful feature to have.
-Synchronization w.r.to user localization needs to be looked at. 
-- It is possible right now that when user localization is in progress for a user, another
task for the same user could get launched before the localization completes. 
-- Also, the object on which we are locking - is it guaranteed that it is a unique instance
for every user ?
- Race condition exists between creation and deletion of user directories. Say a job requires
a user dir and has not yet localized files (and consequently hasn't acquired the synchronization
lock. At that time if deletion starts, it could delete the user dir.
- Also, I think it will be good to check for cleaning up user directories on a much slower
pace as they involve some costly operations.
- I think JobConf.setUserAndGroupNamesForJob need not be static. Also, it would be nice to
document that this is mainly used in test cases.
- User directory can be 570. So also distributed cache directory (no need even for setuid,
right ?)
- The changes in MAPREDUCE-871 need to be synced up in this patch as well.
- Some tests like TestTaskControllerSetup are disabled. Can you please enable them back.
- Permission checks for user directory and jobcache and archive directory permissions needed.
- Test cases should also confirm directory paths in localized distributed cache paths are
being set to the right permissions. 
- Can we use testManagerFlow to have templates that can be overridden by the LinuxTaskController
test class.

> Localized files from DistributedCache should have right access-control
> ----------------------------------------------------------------------
>
>                 Key: MAPREDUCE-856
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-856
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: tasktracker
>            Reporter: Arun C Murthy
>            Assignee: Vinod K V
>         Attachments: MAPREDUCE-856-20090820.txt, MAPREDUCE-856-20090821.txt
>
>


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message