hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hemanth Yamijala (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-896) Users can set non-writable permissions on temporary files for TT and can abuse disk usage.
Date Thu, 03 Dec 2009 06:23:20 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12785200#action_12785200
] 

Hemanth Yamijala commented on MAPREDUCE-896:
--------------------------------------------

Started looking at this patch. I need to still see the C changes, but have the following comments
so far:

- I think we should not pass the full path for deletion to the task-controller for security
purposes. We should use the pattern we have used for other commands like this:
 task-controller user-name cmd mapred-local-dir job-id task-id is-work-dir
- PathDeletionContext can have APIs like jobId, taskId, isWorkDir, which it can construct
based on the path passed to it.
- Can we call the API and commands in TaskController as cleanupTask, just so the names are
consistent like initializeTask, etc.
- The new addToQueue API can be package private.
- We should probably take care that if user or taskController is not null, the other is non-null
as well and throw an IllegalArgumentException if it is not.
- After enabling path for deletion using taskcontroller, we should probably use context.fs.delete(context.path,
true). This way the deletion logic remains same. Also, the deletePath implementation can be
simplified as follows:
{code}
  if (context.taskController != null) {
    context.taskController.cleanupTask(context); 
  }
  return context.fs.delete(context.path, true);
{code}
- I don't think LinuxTaskController must do context.fs.delete in case arguments in pathdeletioncontext
are incorrect. This can hide errors and result in the undeleted directories.
- There is a change in JVMManager where the workdir for the last task was being deleted inline,
but now we delete it asynchronously. Is this OK ? It seems like it should be because there
is no reuse of directories after this point and hence, a little delay in cleaning up the workdir
shouldn't have any effect.
- The change in setupWorkDir does not seem to be creating a workDir after deletion.

> Users can set non-writable permissions on temporary files for TT and can abuse disk usage.
> ------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-896
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-896
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: tasktracker
>    Affects Versions: 0.21.0
>            Reporter: Vinod K V
>            Assignee: Ravi Gummadi
>             Fix For: 0.21.0
>
>         Attachments: MR-896.patch, MR-896.v1.patch
>
>
> As of now, irrespective of the TaskController in use, TT itself does a full delete on
local files created by itself or job tasks. This step, depending upon TT's umask and the permissions
set by files by the user, for e.g in job-work/task-work or child.tmp directories, may or may
not go through successful completion fully. Thus is left an opportunity for abusing disk space
usage either accidentally or intentionally by TT/users.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message