hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rasit OZDAS <rasitoz...@gmail.com>
Subject Re: Hadoop setup questions
Date Sat, 14 Feb 2009 00:34:00 GMT
With this configuration, any user having that group name will be able
to write to any location..
(I've tried this in local network, though)


2009/2/14 Rasit OZDAS <rasitozdas@gmail.com>:
> I agree with Amar and James,
>
> if you require permissions for your project,
> then
> 1. create a group in linux for your user.
> 2. give group write access to all files in HDFS. (hadoop dfs -chmod -R
> g+w /  - or sth, I'm not totally sure.)
> 3. change group ownership of all files in HDFS. (hadoop dfs -chgrp -R
> <your_group_name> /       - I'm not totally sure again..)
>
> cheers,
> Rasit
>
>
> 2009/2/12 james warren <james@rockyou.com>:
>> Like Amar said.  Try adding
>>
>> <property>
>> <name>dfs.permissions</name>
>> <value>false</value>
>> </property>
>>
>>
>> to your conf/hadoop-site.xml file (or flip the value in hadoop-default.xml),
>> restart your daemons and give it a whirl.
>>
>> cheers,
>> -jw
>>
>> On Wed, Feb 11, 2009 at 8:44 PM, Amar Kamat <amarrk@yahoo-inc.com> wrote:
>>
>>> bjday wrote:
>>>
>>>> Good morning everyone,
>>>>
>>>> I have a question about correct setup for hadoop.  I have 14 Dell
>>>> computers in a lab.   Each connected to the internet and each independent
of
>>>> each other.  All run CentOS.  Logins are handled by NIS.  If userA logs
into
>>>> the master and starts the daemons and UserB logs into the master and wants
>>>> to run a job while the daemons from UserA are still running the following
>>>> error occurs:
>>>>
>>>> copyFromLocal: org.apache.hadoop.security.AccessControlException:
>>>> Permission denied: user=UserB, access=WRITE,
>>>> inode="user":UserA:supergroup:rwxr-xr-x
>>>>
>>> Looks like one of your files (input or output) is of different user. Seems
>>> like your DFS has permissions enabled. If you dont require permissions then
>>> disable it else make sure that the input/output paths are under your
>>> permission (/user/userB is the hone directory for userB).
>>> Amar
>>>
>>>
>>>> what needs to be changed to allow UserB-UserZ to run their jobs?  Does
>>>> there need to be a local user the everyone logs into as and run from there?
>>>>  Should Hadoop be ran in an actual cluster instead of independent computers?
>>>>  Any ideas what is the correct configuration settings that allow it?
>>>>
>>>> I followed Ravi Phulari suggestions and followed:
>>>>
>>>> http://hadoop.apache.org/core/docs/current/quickstart.html
>>>>
>>>> http://www.michael-noll.com/wiki/Running_Hadoop_On_Ubuntu_Linux_(Multi-Node_Cluster)<
>>>> http://www.michael-noll.com/wiki/Running_Hadoop_On_Ubuntu_Linux_%28Multi-Node_Cluster%29>
>>>>
>>>>
>>>> http://www.michael-noll.com/wiki/Running_Hadoop_On_Ubuntu_Linux_(Single-Node_Cluster)<
>>>> http://www.michael-noll.com/wiki/Running_Hadoop_On_Ubuntu_Linux_%28Single-Node_Cluster%29>
>>>>
>>>>
>>>> These allowed me to get Hadoop running on the 14 computers when I login
>>>> and everything works fine, thank you Ravi.  The problem occurs when
>>>> additional people attempt to run jobs simultaneously.
>>>>
>>>> Thank you,
>>>>
>>>> Brian
>>>>
>>>>
>>>
>>
>
>
>
> --
> M. Raşit ÖZDAŞ
>



-- 
M. Raşit ÖZDAŞ

Mime
View raw message