hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod K V (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4491) Per-job local data on the TaskTracker node should have right access-control
Date Wed, 22 Jul 2009 11:45:15 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12734079#action_12734079

Vinod K V commented on HADOOP-4491:

Broadly, there are two directory strucutres - system and users

 - system directory will be owned by mapreduce, thereby protecting the contents.
 - users is 755, owned by mapreduce
 - users/$jobid is clearly 700 and owned by the user.

 - system/$jobid/outputs can be directly $ttroot/ as was discussed offline. But I've left
it inside system/$jobid as the $jobid directory seemed reduntant to me. In any case, the outputs
once moved need to owned by the TT.

 - all of the files localized by the TT are written into system/$jobid
 - After job localization is done, all files under system/$jobid/userfiles are moved to users/$jobid
to be consumed by the user's task and so owned by the user.
 - After task localization is done, the whole directory system/$jobid/$taskid is moved to
users/$jobid/ and owned by the user.

 - when the task finishes, the whole users/$user/$jobid/$attemptid/output directory needs
to be moved to outputs/$jobid/$attemptid.

 - cleaning up of a task is removal of  users//$jobid/$attemptid
 - cleaning up a job is removal of users//$jobid, system/$jobid

These changs will be needed for both DefaultTaskController as well as the LinuxTaskController.

LinuxTaskController uses the setuid binary to do the move operations as the root and changing
ownership of the target files to the user.

Distributed cache files and the log files still need to be baked into this structure.

> Per-job local data on the TaskTracker node should have right access-control
> ---------------------------------------------------------------------------
>                 Key: HADOOP-4491
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4491
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: security
>            Reporter: Arun C Murthy
>            Assignee: Vinod K V
>         Attachments: HADOOP-4491-20090623-common.1.txt, HADOOP-4491-20090623-mapred.1.txt,
HADOOP-4491-20090703-common.1.txt, HADOOP-4491-20090703-common.txt, HADOOP-4491-20090703.1.txt,
HADOOP-4491-20090703.txt, HADOOP-4491-20090707-common.txt, HADOOP-4491-20090707.txt, HADOOP-4491-20090716-mapred.txt

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message