hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amar Kamat (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1422) Changing permissions of files/dirs under job-work-dir may be needed sothat cleaning up of job-dir in all mapred-local-directories succeeds always
Date Fri, 05 Mar 2010 09:10:28 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12841766#action_12841766
] 

Amar Kamat commented on MAPREDUCE-1422:
---------------------------------------

Hemanth,
I am confused w.r.t point #3 and #4. I added test-code in TestTaskTrackerLocalization for
testing job-work-dir cleanup because there is a call to _tracker.removeJobFiles()_. Now  TestLocalizationWithLinuxTaskController
(that extends TestTaskTrackerLocalization) will also test the job-work-dir cleanup code with
LinuxTaskController. As you are suggesting, if we move this job cleanup test-code into a separate
testcase, then we need TestJobLocalDirCleanupWithLinuxTaskController, no?

Another way to do this would be to introduce a new testcase (say TestJobDirCleanup) as you
suggested which will test the job-dir and task-dirs upon job completion. Also add a  testcase
(say TestJobDirCleanupWithLinuxTaskController)  extending TestJobDirCleanup but configured
with LinuxTaskController. Thoughts?

> Changing permissions of files/dirs under job-work-dir may be needed sothat cleaning up
of job-dir in all mapred-local-directories succeeds always
> -------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1422
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1422
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: tasktracker
>            Reporter: Ravi Gummadi
>            Assignee: Amar Kamat
>         Attachments: mapreduce-1422-test-v1.0.patch, mapreduce-1422-v1.4.2.patch
>
>
> After MAPREDUCE-896, if LinuxTaskController is set in config, task-controller binary
is launched for changing permissions of taskAttemptDir and taskWorkDir before cleaning up
of these directories sothat cleanup will be succeeded even if user had created files/dirs
under taskAttemptDir or taskWorkDir with non-writable permissions. Users can't create files/dirs
under job-dir directly as we set 2570 for job-dir. But as job-work-dir has 2770 permissions
and user can create files/dirs under job-work-dir with non-writable permissions, Changing
permissions of files/dirs under job-work-dir may be needed sothat cleaning up of job-dir in
all mapred-local-directories succeeds always.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message