hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hemanth Yamijala (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-896) Users can set non-writable permissions on temporary files for TT and can abuse disk usage.
Date Tue, 15 Dec 2009 16:12:18 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12790791#action_12790791
] 

Hemanth Yamijala commented on MAPREDUCE-896:
--------------------------------------------

The trunk patch is looking close. Have a few minor comments:

- Seems like getLocalDirs() can be implemented in terms of getLocalFiles() by calling getLocalFiles(fConf,
"") and adjusting getLocalFiles to handle the case where if the 'subDir' is empty, it works
by returning just the local files.
- Use TaskTracker.getTaskWorkDir in TaskControllerPathDeletionContext.buildPathForDeletion().
Same for LinuxTaskController.buildTaskCleanupArgs.
- Remove the TODO in testSetupWorkDir. Instead we could file a new JIRA to track the additional
test cases.
- Couple of more test cases I can think of:
-- Call removeTaskFiles with needCleanup set to false, and verify the entire task directory
is cleaned up.
-- Call removeTaskFiles with jobconf enabling JVM reuse and verify the work directory is cleaned
up.
-- Do we need to ensure changes in JobTracker and JobInProgress are tested too ? Or are they
already covered elsewhere ?

> Users can set non-writable permissions on temporary files for TT and can abuse disk usage.
> ------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-896
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-896
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: tasktracker
>    Affects Versions: 0.21.0
>            Reporter: Vinod K V
>            Assignee: Ravi Gummadi
>             Fix For: 0.21.0
>
>         Attachments: MR-896.patch, MR-896.v1.patch, MR-896.v2.patch, y896.v1.patch, y896.v2.1.patch,
y896.v2.patch
>
>
> As of now, irrespective of the TaskController in use, TT itself does a full delete on
local files created by itself or job tasks. This step, depending upon TT's umask and the permissions
set by files by the user, for e.g in job-work/task-work or child.tmp directories, may or may
not go through successful completion fully. Thus is left an opportunity for abusing disk space
usage either accidentally or intentionally by TT/users.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message