hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod K V (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4491) Per-job local data on the TaskTracker node should have right access-control
Date Tue, 23 Jun 2009 08:26:07 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12723003#action_12723003
] 

Vinod K V commented on HADOOP-4491:
-----------------------------------

Few points: 
 - After discussing with Hemanth/Sreekanth and looking up the original intentions of HADOOP-4490,
I've reworked the above proposal to have very tight(0700) permissions everywhere.
 - But to allow multiple users to create paths, the following paths still have 755 permissions:
taskTracker, jobcache, job directory, root-log-dir and distribute cache directory.
 - We need more work to secure the logs. In these patches, log directories of attempts inside
userlogs are owned by the user, but still have 755 permissions fo the TT to read them back
to serve via TaskLogServlet.

Here's what the patches do:

common-patch:
 - Changes LocalDirAllocator and DiskChecker to changes permissions of the paths created using
java api to set permission instead of spawning chmod processes.

mapreduce-patch:
 - Changes TaskTracker related classes to create secure paths with 700 permissions by calling
LocalDirAllocator.getSecurePathForWrite() instead of the usual LocalDirAllocator.getLocalPathForWrite()
 - Splits TaskTracker.getIntermediateOutputDir() into TaskTracker.getIntermediateOutputDirForChild
and TaskTracker.getIntermediateOutputDirForTT() as now, child's mapred.local.dir is modified
so as to sandbox it for preserving secure permissions.
 - Makes changes to LinuxTaskController binary to do the following:
   -- Change ownership of attempt-dir, jars-dir, log-dir to be owned by the child at jvm startup.
   -- Finalize task-directories to be owned back by the TT when a task finishes for output
serving and cleaning up.
   -- Finalizing job-dirs to be owned by the TT when the job finishes.
 - Adds a new c based testing for the LinuxTaskController and makes relevant changes to the
build.xml, MakeFile, configure.ac. This test can be run by the target test-task-controller.
 - Modifies tests inheriting from ClusterWithLinuxTaskController to reflect the above changes.
 - Incorporates changes needed for MAPREDUCE-131.

I've tested the patch by running the LinuxTaskController specific tests and observing the
directory structure over time. All the tests with successful, failed, killed jobs, with and
without reuse pass in the sense that throughout a job's timeline, any path is either owned
by the TT or the child and has the most secure permissions possible.

> Per-job local data on the TaskTracker node should have right access-control
> ---------------------------------------------------------------------------
>
>                 Key: HADOOP-4491
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4491
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: mapred, security
>            Reporter: Arun C Murthy
>            Assignee: Vinod K V
>         Attachments: HADOOP-4491-20090623-common.1.txt, HADOOP-4491-20090623-mapred.1.txt
>
>


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message