hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ravi Gummadi (JIRA)" <j...@apache.org>
Subject [jira] Updated: (MAPREDUCE-896) Users can set non-writable permissions on temporary files for TT and can abuse disk usage.
Date Mon, 16 Nov 2009 07:16:39 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Ravi Gummadi updated MAPREDUCE-896:

    Attachment: MR-896.patch

Attaching patch with the fix. Please review and provide your comments.

Patch does the following:

(1) When deleting $jobId/$attemptId/work or when deleting $jobId/$attemptId, TT uses task-controller
to enable the path for deletion(by changing permissions). (a) LinuxTaskController sets 770
as permissions for all the files/directories within this dir recursively and then TT will
delete the dir. (b) DefaultTaskController sets rwx for user(same as TT) for this dir recursively
and then TT deletes the dir.

(2) Deletion of $jobId is done as earlier because user can't create any files in this dir.

(3) Deletion of work dir in TaskRunner(useful with jvm reuse) is also done by changing the
permissions first(no taskcontroller is needed in this case).

(4) With jvm reuse, after final task of a Jvm is finished, workDir is deleted using procedure
mentioned in (1) above by task-controller.

(5) Modified TestTaskTrackerLocalization and TestLocalizationWithLinuxTaskController to have
directories/files created in $attemptId and set nonwritable permissions which tests whether
(a) DefaultTaskController and (b) LinuxTaskController would be able to remove these new files
from $attemptId.

> Users can set non-writable permissions on temporary files for TT and can abuse disk usage.
> ------------------------------------------------------------------------------------------
>                 Key: MAPREDUCE-896
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-896
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: tasktracker
>    Affects Versions: 0.21.0
>            Reporter: Vinod K V
>            Assignee: Ravi Gummadi
>             Fix For: 0.21.0
>         Attachments: MR-896.patch
> As of now, irrespective of the TaskController in use, TT itself does a full delete on
local files created by itself or job tasks. This step, depending upon TT's umask and the permissions
set by files by the user, for e.g in job-work/task-work or child.tmp directories, may or may
not go through successful completion fully. Thus is left an opportunity for abusing disk space
usage either accidentally or intentionally by TT/users.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message