hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hemanth Yamijala (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1186) While localizing a DistributedCache file, TT sets permissions recursively on the whole base-dir
Date Mon, 09 Nov 2009 14:38:35 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12774989#action_12774989
] 

Hemanth Yamijala commented on MAPREDUCE-1186:
---------------------------------------------

Here's a proposal:
- We should remove the recursive call to set permissions in TrackerDistributedCacheManager.localizeCache().
- To handle the case for the default task controller, we can set execute permissions for all
localized files and archives as done before HADOOP-4490. This can be done by using Java APIs
directly rather than executing chmod.
- For the LinuxTaskController, we can take the help of changes introduced in MAPREDUCE-1098.
-- In MAPREDUCE-1098, a random id was generated into the local file path for every URI that
was freshly localized.
-- So I suppose we can assume that if a random id has the right permissions and ownership,
all files under this directory would have the right permissions and ownership. Conversely,
if these are not set, we should set them for all files under this directory.
-- We can get the random id directory by removing the normalized URI path from the full path
saved in the task's configuration for a given URI.

Does this work ?

> While localizing a DistributedCache file, TT sets permissions recursively on the whole
base-dir
> -----------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1186
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1186
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: tasktracker
>    Affects Versions: 0.21.0
>            Reporter: Vinod K V
>             Fix For: 0.21.0
>
>
> This is a performance problem.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message