hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hemanth Yamijala (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4490) Map and Reduce tasks should run as the user who submitted the job
Date Sun, 01 Feb 2009 08:27:59 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12669354#action_12669354

Hemanth Yamijala commented on HADOOP-4490:

I've updated the patch to trunk, incorporating most of Arun's comments above. Arun, can you
please take a look.

bq. We should use mapred.local.dir instead of hadoop.tmp.dir in LinuxTaskController.

bq. Use Path's methods instead of String manipulation for all path-related manipulations.

bq. Pass mode, user/group to DistributedCache rather than rely on the newly introduced DistributedCache.isFreshlyLoaded
which is then unnecessary.
Done. I've added a new overloaded API that passes the information to DistributedCache. Just
to keep options open, I've defined a new public class DistributedCacheFileAccessInfo - a simple
class that can be used to define permissions and ownership information for localized files
in DistributedCache. Can you take a specific look at this, and let me know if this looks OK

bq. Move setting up of JVM-specific files e.g. task's log directory to TaskController.launchJVM
I've not done this one alone. It was not very clear what information is necessary at launch
time. For e.g. if there are some localized files under the task cache directory that need
to be loaded at launch time, we'll need permissions for these also. In general, it seemed
a little risky to launch the JVM without giving full access to all jars etc, even if the Task
will start running later only. So, I've left this as is. I think the main concern here was
about the special check I had in JvmManager where I was avoiding setting the permissions again
when getting the task to launch. This seems a simple enough check, and I've documented the
rationale in code. Can you verify this again, and let me know your thoughts ?

> Map and Reduce tasks should run as the user who submitted the job
> -----------------------------------------------------------------
>                 Key: HADOOP-4490
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4490
>             Project: Hadoop Core
>          Issue Type: Sub-task
>          Components: mapred, security
>            Reporter: Arun C Murthy
>            Assignee: Hemanth Yamijala
>         Attachments: hadoop-4490-design.pdf, HADOOP-4490.patch, HADOOP-4490.patch, HADOOP-4490.patch,
> Currently the TaskTracker spawns the map/reduce tasks, resulting in them running as the
user who started the TaskTracker.
> For security and accounting purposes the tasks should be run as the job-owner.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message