hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From stephen mulcahy <stephen.mulc...@deri.org>
Subject hadoop multi-user cluster
Date Wed, 09 Nov 2011 18:13:07 GMT

I've just installed a small hadoop cluster (4 nodes) and am 
trying to configure it such that different users can run jobs on it 
(rather than having everyone submit jobs as the superuser).

Using a pretty standard config, I've found I need to make the following 
changes before users can successfully submit jobs.

hadoop/bin/hadoop fs -chmod 777 /tmp/hadoop/mapred/staging


hadoop/bin/hadoop fs -chmod 777 /hadoop/mapred/system

where /hadoop/mapred/system is my mapred.system.dir

The second one seems to be required everytime I restart the cluster.

Are these to be expected on a multi-user cluster or am I missing something?

If these aren't specified, user jobs fail with an Exception like

11/11/09 16:32:53 INFO mapred.FileInputFormat: Total input paths to 
process : 2
11/11/09 16:32:53 INFO mapred.JobClient: Running job: job_201111091731_0003
11/11/09 16:32:54 INFO mapred.JobClient:  map 0% reduce 0%
11/11/09 16:32:54 INFO mapred.JobClient: Job complete: job_201111091731_0003
11/11/09 16:32:54 INFO mapred.JobClient: Counters: 0
11/11/09 16:32:54 INFO mapred.JobClient: Job Failed: Job initialization 
org.apache.hadoop.security.AccessControlException: Permission denied: 
user=smulcahy, access=EXECUTE, inode="system":hadoop:supergroup:rwx------



Stephen Mulcahy, DI2, Digital Enterprise Research Institute,
NUI Galway, IDA Business Park, Lower Dangan, Galway, Ireland
http://di2.deri.ie    http://webstar.deri.ie    http://sindice.com

View raw message