hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hemanth Yamijala (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4490) Map and Reduce tasks should run as the user who submitted the job
Date Fri, 13 Mar 2009 13:15:51 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12681701#action_12681701
] 

Hemanth Yamijala commented on HADOOP-4490:
------------------------------------------

One more change required. I've noticed that when the JVM Manager calls a killTaskJVM, the
current working directory for the LinuxTaskController is set to a task attempt directory.
This will fail, because this directory will no longer exist. While this is not creating problems
as well behaved JVMs will exit themselves, it is not something to be relied on.

The relevant piece of code is this:

{code}
    private ShellCommandExecutor buildTaskControllerExecutor(TaskCommands command, 
                                            String userName, 
                                            List<String> cmdArgs, JvmEnv env) 
                                              throws IOException {

      ...
      if (env != null) {
       shExec = new ShellCommandExecutor(taskControllerCmd,
            env.workDir, env.env);
      }else {
        shExec = new ShellCommandExecutor(taskControllerCmd);
      }
    }
{code}

Setting env.workDir will set the working directory to that particular task attempt directory
which may no longer exist. I get the following error when this happens:

2009-03-13 10:39:52,750 WARN org.apache.hadoop.mapred.LinuxTaskController: IOException in
killing task: Cannot run program "/path/to/bin/task-controller" (in directory      "/path/to/mapred-local/taskTracker/jobcache/job_200903130908_0051/attempt_200903130908_0051_m_001012_1000/work"):
java.io.IOException: error=2, No such file or directory


    


> Map and Reduce tasks should run as the user who submitted the job
> -----------------------------------------------------------------
>
>                 Key: HADOOP-4490
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4490
>             Project: Hadoop Core
>          Issue Type: Sub-task
>          Components: mapred, security
>            Reporter: Arun C Murthy
>            Assignee: Hemanth Yamijala
>             Fix For: 0.21.0
>
>         Attachments: HADOOP-4490-1.patch, HADOOP-4490-1.patch, hadoop-4490-design.pdf,
HADOOP-4490.patch, HADOOP-4490.patch, HADOOP-4490.patch, HADOOP-4490.patch, HADOOP-4490.patch,
HADOOP-4490.patch, HADOOP-4490.patch, HADOOP-4490_streaming.patch
>
>
> Currently the TaskTracker spawns the map/reduce tasks, resulting in them running as the
user who started the TaskTracker.
> For security and accounting purposes the tasks should be run as the job-owner.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message