hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hemanth Yamijala (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1422) Changing permissions of files/dirs under job-work-dir may be needed sothat cleaning up of job-dir in all mapred-local-directories succeeds always
Date Tue, 09 Mar 2010 18:29:30 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12843223#action_12843223
] 

Hemanth Yamijala commented on MAPREDUCE-1422:
---------------------------------------------

The latest patch looks good to me. +1.

Note though that:

bq. added checks to make sure that the job-work-dirs/task-local-dirs contains atleast one
path

This is not checking the existence of paths, the returned array is simply a list of path objects
created with no bearing on the physical directory / file creation. However, I think this is
still fine because calling localizeJob ensures work directory is created.... this is tested
in testJobLocalization(). Likewise, calling localizeTask ensures attempt directory is created..
this is tested in testTaskLocalization. So, I think it is OK even to not check these conditions
in the new test method.

> Changing permissions of files/dirs under job-work-dir may be needed sothat cleaning up
of job-dir in all mapred-local-directories succeeds always
> -------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1422
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1422
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: tasktracker
>            Reporter: Ravi Gummadi
>            Assignee: Amar Kamat
>         Attachments: mapreduce-1422-test-v1.0.patch, mapreduce-1422-v1.4.2.patch, mapreduce-1422-v1.5.2.patch,
mapreduce-1422-v1.5.4.patch, mapreduce-1422-v1.5.5.patch
>
>
> After MAPREDUCE-896, if LinuxTaskController is set in config, task-controller binary
is launched for changing permissions of taskAttemptDir and taskWorkDir before cleaning up
of these directories sothat cleanup will be succeeded even if user had created files/dirs
under taskAttemptDir or taskWorkDir with non-writable permissions. Users can't create files/dirs
under job-dir directly as we set 2570 for job-dir. But as job-work-dir has 2770 permissions
and user can create files/dirs under job-work-dir with non-writable permissions, Changing
permissions of files/dirs under job-work-dir may be needed sothat cleaning up of job-dir in
all mapred-local-directories succeeds always.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message