hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hemanth Yamijala (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1186) While localizing a DistributedCache file, TT sets permissions recursively on the whole base-dir
Date Thu, 24 Dec 2009 11:32:29 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12794392#action_12794392
] 

Hemanth Yamijala commented on MAPREDUCE-1186:
---------------------------------------------

This is getting close. I do have some feedback though:

- In the same line as removing the dead call to cleanupStorage(), should we also remove the
call to this.distributedCacheManager.purgeCache() ?
- I don't think we need to convert the random number to a positive number. Doing so actually
increases the chances our random numbers could clash with previous calls. Offline, I actually
just found out why you did this (*smile*). So, let's fix the other issue by passing an escape
sequence for the negative number to the task controller.
- Should we move the logic to construct the full path to the distributed cache file into DistributedCacheFileContext,
in some method like getLocalizedUniqueDirectory ?
- Previously we were setting ugo+rx for distributed cache files. In DefaultTaskController,
we are changing this to only execute. I suggest we not change this. Any reason for doing so
?
- Rename TaskCommands.INITIALIZE_DISTRIBUTEDCACHE also to TaskCommands.INITIALIZE_DISTRIBUTEDCACHEFILE.
- localized_unique_dir in task-controller.c should be freed to release memory.
- Instead of using the 'failed' variable as a boolean, I think you can directly set 'failed'
to INITIALIZE_DISTCACHE_FAILED and return it.
- Please add documentation to describe testCustomPermissions.
- In testCustomPermissions, delete the hand created file in a finally block to handle cleanup
during unexpected failures. 

> While localizing a DistributedCache file, TT sets permissions recursively on the whole
base-dir
> -----------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1186
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1186
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: tasktracker
>    Affects Versions: 0.21.0
>            Reporter: Vinod K V
>            Assignee: Amareshwari Sriramadasu
>             Fix For: 0.22.0
>
>         Attachments: patch-1186-1.txt, patch-1186-2.txt, patch-1186-3-ydist.txt, patch-1186-3-ydist.txt,
patch-1186-3.txt, patch-1186-ydist.txt, patch-1186-ydist.txt, patch-1186.txt
>
>
> This is a performance problem.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message