hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-181) Secure job submission
Date Tue, 08 Sep 2009 21:41:58 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12752759#action_12752759

Owen O'Malley commented on MAPREDUCE-181:

Instead of storing the UGI with the submitted job, please store the user as a string. That
will be forward compatible when we move to server-side groups. I think it makes sense to do
as part of this patch, if it isn't already being done.

The meta information should only include the offset, since the length is redundant with the
following split's start.

We use the binary format instead of xml to store the jobconf. However, when loading the binary
format, we need to handle the final parameters.

I'm not very happy with half of the job information being saved in the system directory and
half of it in the staging directory. I assume that the staging directory is required to be
on the same file system as the system directory? Having the job's definition split into two
directories with two different owners seems bad. That is especially true since the data in
the system directory will point to particular byte offsets in the staging directory. I think
we will be in for some really nasty bugs involving 

I assume the cleanup of the staging directory is done by the JobTracker.

I guess I would be happier, if as part of JobSubmission, we moved the files from the user's
staging area into the system dir. The JobTracker would read (possibly with a cache) the bytes
for the task and send them to the user as part of the task definition.

> Secure job submission 
> ----------------------
>                 Key: MAPREDUCE-181
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-181
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Amar Kamat
>            Assignee: Amar Kamat
>         Attachments: hadoop-3578-branch-20-example-2.patch, hadoop-3578-branch-20-example.patch,
HADOOP-3578-v2.6.patch, HADOOP-3578-v2.7.patch, MAPRED-181-v3.8.patch
> Currently the jobclient accesses the {{mapred.system.dir}} to add job details. Hence
the {{mapred.system.dir}} has the permissions of {{rwx-wx-wx}}. This could be a security loophole
where the job files might get overwritten/tampered after the job submission. 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message