hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From stephen mulcahy <stephen.mulc...@deri.org>
Subject hadoop 0.20.205.0 multi-user cluster
Date Wed, 09 Nov 2011 18:13:07 GMT
Hi,

I've just installed a small hadoop 0.20.205.0 cluster (4 nodes) and am 
trying to configure it such that different users can run jobs on it 
(rather than having everyone submit jobs as the superuser).

Using a pretty standard config, I've found I need to make the following 
changes before users can successfully submit jobs.

hadoop/bin/hadoop fs -chmod 777 /tmp/hadoop/mapred/staging

and

hadoop/bin/hadoop fs -chmod 777 /hadoop/mapred/system

where /hadoop/mapred/system is my mapred.system.dir

The second one seems to be required everytime I restart the cluster.

Are these to be expected on a multi-user cluster or am I missing something?

If these aren't specified, user jobs fail with an Exception like

11/11/09 16:32:53 INFO mapred.FileInputFormat: Total input paths to 
process : 2
11/11/09 16:32:53 INFO mapred.JobClient: Running job: job_201111091731_0003
11/11/09 16:32:54 INFO mapred.JobClient:  map 0% reduce 0%
11/11/09 16:32:54 INFO mapred.JobClient: Job complete: job_201111091731_0003
11/11/09 16:32:54 INFO mapred.JobClient: Counters: 0
11/11/09 16:32:54 INFO mapred.JobClient: Job Failed: Job initialization 
failed:
org.apache.hadoop.security.AccessControlException: 
org.apache.hadoop.security.AccessControlException: Permission denied: 
user=smulcahy, access=EXECUTE, inode="system":hadoop:supergroup:rwx------
.....


Thanks,

-stephen

-- 
Stephen Mulcahy, DI2, Digital Enterprise Research Institute,
NUI Galway, IDA Business Park, Lower Dangan, Galway, Ireland
http://di2.deri.ie    http://webstar.deri.ie    http://sindice.com

Mime
View raw message