hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rasit OZDAS <rasitoz...@gmail.com>
Subject Re: Hadoop setup questions
Date Sat, 14 Feb 2009 00:31:01 GMT
I agree with Amar and James,

if you require permissions for your project,
then
1. create a group in linux for your user.
2. give group write access to all files in HDFS. (hadoop dfs -chmod -R
g+w /  - or sth, I'm not totally sure.)
3. change group ownership of all files in HDFS. (hadoop dfs -chgrp -R
<your_group_name> /       - I'm not totally sure again..)

cheers,
Rasit


2009/2/12 james warren <james@rockyou.com>:
> Like Amar said.  Try adding
>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
>
>
> to your conf/hadoop-site.xml file (or flip the value in hadoop-default.xml),
> restart your daemons and give it a whirl.
>
> cheers,
> -jw
>
> On Wed, Feb 11, 2009 at 8:44 PM, Amar Kamat <amarrk@yahoo-inc.com> wrote:
>
>> bjday wrote:
>>
>>> Good morning everyone,
>>>
>>> I have a question about correct setup for hadoop.  I have 14 Dell
>>> computers in a lab.   Each connected to the internet and each independent of
>>> each other.  All run CentOS.  Logins are handled by NIS.  If userA logs into
>>> the master and starts the daemons and UserB logs into the master and wants
>>> to run a job while the daemons from UserA are still running the following
>>> error occurs:
>>>
>>> copyFromLocal: org.apache.hadoop.security.AccessControlException:
>>> Permission denied: user=UserB, access=WRITE,
>>> inode="user":UserA:supergroup:rwxr-xr-x
>>>
>> Looks like one of your files (input or output) is of different user. Seems
>> like your DFS has permissions enabled. If you dont require permissions then
>> disable it else make sure that the input/output paths are under your
>> permission (/user/userB is the hone directory for userB).
>> Amar
>>
>>
>>> what needs to be changed to allow UserB-UserZ to run their jobs?  Does
>>> there need to be a local user the everyone logs into as and run from there?
>>>  Should Hadoop be ran in an actual cluster instead of independent computers?
>>>  Any ideas what is the correct configuration settings that allow it?
>>>
>>> I followed Ravi Phulari suggestions and followed:
>>>
>>> http://hadoop.apache.org/core/docs/current/quickstart.html
>>>
>>> http://www.michael-noll.com/wiki/Running_Hadoop_On_Ubuntu_Linux_(Multi-Node_Cluster)<
>>> http://www.michael-noll.com/wiki/Running_Hadoop_On_Ubuntu_Linux_%28Multi-Node_Cluster%29>
>>>
>>>
>>> http://www.michael-noll.com/wiki/Running_Hadoop_On_Ubuntu_Linux_(Single-Node_Cluster)<
>>> http://www.michael-noll.com/wiki/Running_Hadoop_On_Ubuntu_Linux_%28Single-Node_Cluster%29>
>>>
>>>
>>> These allowed me to get Hadoop running on the 14 computers when I login
>>> and everything works fine, thank you Ravi.  The problem occurs when
>>> additional people attempt to run jobs simultaneously.
>>>
>>> Thank you,
>>>
>>> Brian
>>>
>>>
>>
>



-- 
M. Raşit ÖZDAŞ

Mime
View raw message