hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hairong Kuang <hair...@yahoo-inc.com>
Subject Re: mapreduce does the wrong thing with dfs permissions?
Date Tue, 26 Feb 2008 21:17:20 GMT
Before you file a jira, could you please post the error message? Let us
check what went wrong in your case. The design is that every job talks to
dfs as the user who submitted the job. Please check
https://issues.apache.org/jira/browse/HADOOP-1873 for more information on
user permissions and mapred.


On 2/26/08 12:08 PM, "Michael Bieniosek" <michael@powerset.com> wrote:

> This is not the behavior I was seeing -- to use your example, the tasktracker
> tried to talk to the the DFS as the "foo" user, not the "bar" user who
> submitted the job.  Should I file a JIRA then?
> -Michael
> On 2/26/08 11:13 AM, "s29752-hadoopdev@yahoo.com" <s29752-hadoopdev@yahoo.com>
> wrote:
>> The problem is that the tasktrackers always run under the same UNIX account,
>> "mapreduce".  I can submit a job as "user", but the tasktracker will still
>> talk to the dfs as the "mapreduce" user.  This means that everything that
>> hadoop mapreduce touches has to be owned in the dfs by the "mapreduce" user.
>> If everything is owned and run by the same user, then permissions are
>> pointless.
> I am not quite understand your situation but the tasktracker account should
> not matter.  Suppose a tasktracker is ran by foo and a job is submitted by
> bar.  Then, the permission checking during the execution of the job is against
> the job submitter (bar), not tasktracker (foo).  In your case, if the job is
> submitted by "user" and "user" is able to read the input files and access
> other required files, than you should not get any AccessControlException.
> Nicholas

View raw message