hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From alo alt <wget.n...@googlemail.com>
Subject Re: HDFS mounting issue using Hadoop-Fuse on Fully Distributed Cluster?
Date Thu, 26 Apr 2012 14:23:25 GMT
Manu,

for clarifying:

root has no access to the mounted HDFS. Just follow the howto:

1. create the group and the users on ALL nodes:
groupadd hdfs-user && adduser USERNAME -G hdfs-user 

2. sudo into hdfs:
su - hdfs

3. create a directory in hdfs and change the rights:
hadoop fs -mkdir /someone && hadoop fs -chmod 774 /someone && hadoop fs -chgrp
hdfs-user /someone

Now the users you created and added into to group are able to write files.

- Alex
 

--
Alexander Lorenz
http://mapredit.blogspot.com

On Apr 26, 2012, at 3:58 PM, alo alt wrote:

> Yes, as I wrote. You can't use root as user for writing, root (or superuser) has another
context in hdfs. Just change into hdfs (su - hdfs) and try again. For all user who should
have access to the mounted fs you should create a group and chown them in hdfs (maybe /tmp/group
or similar)
> 
> best,
> Alex 
> 
> 
> On Apr 26, 2012, at 2:53 PM, Manu S wrote:
> 
>> Yeah Alex, I tried. But still I am not able to make it
>> 
>> [root@namenode ~]# echo "hadoop-fuse-dfs#dfs://namenode:8020 /hdfs_mount fuse usetrash,rw
0 0" >> /etc/fstab
>> [root@namenode ~]# mount -a
>> INFO fuse_options.c:162 Adding FUSE arg /hdfs_mount
>> 
>> [root@namenode ~]# mount | grep fuse
>> fuse on /hdfs_mount type fuse (rw,nosuid,nodev,allow_other,default_permissions)
>> 
>> [root@namenode ~]# ls -ld /hdfs_mount/
>> drwxrwxr-x 11 nobody nobody 4096 Apr  9 12:34 /hdfs_mount/
>> [root@namenode ~]# touch /hdfs_mount/file
>> touch: cannot touch `/hdfs_mount/file': Permission denied
>> 
>> 
>> 
>> On Thu, Apr 26, 2012 at 6:09 PM, alo alt <wget.null@googlemail.com> wrote:
>> Manu,
>> 
>> did you mount hdfs over fstab:
>> 
>> hadoop-fuse-dfs#dfs://namenode.local:<PORT> /hdfs-mount fuse usetrash,rw 0
0" ?
>> 
>> You could that do with:
>> "mkdir -p /hdfs-mount && chmod 777 /hdfs-mount && echo "hadoop-fuse-dfs#dfs://NN.URI:<PORT>
/hdfs-mount fuse usetrash,rw 0 0" >> /etc/fstab && mount -a ; mount"
>> 
>> 
>> - Alex
>> 
>> 
>> On Apr 26, 2012, at 2:00 PM, Manu S wrote:
>> 
>>> Thanks a lot Alex.
>>> 
>>> Actually I didn't tried the NFS option, as I am trying to sort out this hadoop-fuse
mounting issue.
>>> I can't change the ownership of mount directory after hadoop-fuse mount.
>>> 
>>> [root@namenode ~]# ls -ld /hdfs_mount/
>>> drwxrwxr-x 11 nobody nobody 4096 Apr  9 12:34 /hdfs_mount/
>>> 
>>> [root@namenode ~]# chown hdfs /hdfs_mount/
>>> chown: changing ownership of `/hdfs_mount/': Input/output error
>>> 
>>> Any ideas?
>>> 
>>> 
>>> On Thu, Apr 26, 2012 at 4:26 PM, alo alt <wget.null@googlemail.com> wrote:
>>> Hi,
>>> 
>>> I wrote a small writeup about:
>>> http://mapredit.blogspot.de/2011/11/nfs-exported-hdfs-cdh3.html
>>> 
>>> As you see, the FS is mounted as nobody and you try as root. Change the permissions
in your hdfs:
>>> hadoop -dfs chmod / chown ....
>>> 
>>> - Alex
>>> 
>>> --
>>> Alexander Lorenz
>>> http://mapredit.blogspot.com
>>> 
>>> On Apr 26, 2012, at 12:50 PM, Manu S wrote:
>>> 
>>>> Dear All,
>>>> 
>>>> I have installed *Hadoop-fuse* to mount the HDFS filesystem locally . I
>>>> could mount the HDFS without any issues.But I am not able to do any file
>>>> operations like *delete, copy, move* etc directly. The directory ownership
>>>> automatically changed to *nobody:nobody* while mounting.
>>>> 
>>>> *[root@namenode ~]# ls -ld /hdfs_mount/
>>>> drwxr-xr-x 2 root root 4096 Apr 3 16:22 /hdfs_mount/
>>>> 
>>>> [root@namenode ~]# hadoop-fuse-dfs dfs://namenode:8020 /hdfs_mount/** **
>>>> INFO fuse_options.c:162 Adding FUSE arg /hdfs_mount/
>>>> 
>>>> [root@namenode ~]# ls -ld /hdfs_mount/* *
>>>> drwxrwxr-x 10 nobody nobody 4096 Apr 5 13:22 /hdfs_mount/*
>>>> 
>>>> I tried the same with *pseudo-distributed node*,but its working fine. I can
>>>> do any normal file operations after mounting the HDFS.
>>>> 
>>>> Appreciate your help on the same.
>>>> 
>>>> --
>>>> Thanks & Regards
>>>> ----
>>>> *Manu S*
>>>> SI Engineer - OpenSource & HPC
>>>> Wipro Infotech
>>>> Mob: +91 8861302855                Skype: manuspkd
>>>> www.opensourcetalk.co.in
>>> 
>>> 
>>> 
>>> 
>>> --
>>> Thanks & Regards
>>> ----
>>> Manu S
>>> SI Engineer - OpenSource & HPC
>>> Wipro Infotech
>>> Mob: +91 8861302855                Skype: manuspkd
>>> www.opensourcetalk.co.in
>>> 
>>> 
>>> 
>> 
>> 
>> --
>> Alexander Lorenz
>> http://mapredit.blogspot.com
>> 
>> 
>> 
>> 
>> -- 
>> Thanks & Regards
>> ----
>> Manu S
>> SI Engineer - OpenSource & HPC
>> Wipro Infotech
>> Mob: +91 8861302855                Skype: manuspkd
>> www.opensourcetalk.co.in
>> 
>> 
>> 
> 
> 
> --
> Alexander Lorenz
> http://mapredit.blogspot.com
> 


Mime
View raw message