hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hemanth Yamijala (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1186) While localizing a DistributedCache file, TT sets permissions recursively on the whole base-dir
Date Tue, 15 Dec 2009 09:21:18 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12790629#action_12790629
] 

Hemanth Yamijala commented on MAPREDUCE-1186:
---------------------------------------------

I was concerned that this will actually result in a process launch for the task-controller
for every localized URI. However, I realized that even pre-HADOOP-4490, there was a process
launch because a chmod was done on the localized file. Given that, I am OK with this proposal.
I would recommend we run some reasonable benchmarks (say a job with lots of files in distributed
cache) with and without the patch a couple of times to see if there's any impact due to this
change.

> While localizing a DistributedCache file, TT sets permissions recursively on the whole
base-dir
> -----------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1186
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1186
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: tasktracker
>    Affects Versions: 0.21.0
>            Reporter: Vinod K V
>            Assignee: Amareshwari Sriramadasu
>             Fix For: 0.21.0
>
>         Attachments: patch-1186-1.txt, patch-1186-3-ydist.txt, patch-1186-3-ydist.txt,
patch-1186-ydist.txt, patch-1186-ydist.txt, patch-1186.txt
>
>
> This is a performance problem.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message