hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From s29752-hadoopu...@yahoo.com
Subject Re: Hadoop Permission Problem
Date Fri, 09 May 2008 20:32:51 GMT
Hi Senthil,

drwxrwxrwx    5 hadoop   hadoop       4096 May  7 18:02 datastore

This one is your local directory.  I think you might have mixed up the local and hdfs directories.

Nicholas



----- Original Message ----
From: "Natarajan, Senthil" <senthil@pitt.edu>
To: "core-user@hadoop.apache.org" <core-user@hadoop.apache.org>
Sent: Friday, May 9, 2008 1:11:01 PM
Subject: RE: Hadoop Permission Problem

Hi Nicholas,
You are right, the permission problem is with datastore, that's what I mentioned in the previous
mails.
But I gave the 777 permission. Here is the datastore permission in the master.

drwxrwxrwx    5 hadoop   hadoop       4096 May  7 18:02 datastore

I am not seeing any datastore in the slave machines.



-----Original Message-----
From: s29752-hadoopuser@yahoo.com [mailto:s29752-hadoopuser@yahoo.com]
Sent: Friday, May 09, 2008 3:29 PM
To: core-user@hadoop.apache.org
Subject: Re: Hadoop Permission Problem

Hi Senthil,

Let me explain the error message " Permission denied: user=test, access=WRITE, inode="datastore":hadoop:supergroup:rwxr-xr-x".
 It says that the current user "test" is trying to WRITE to the inode "datastore" with owner
hadoop:supergroup and permission 755.  So the problem is in the directory "datastore".  Could
you check it?

Nicholas



----- Original Message ----
From: "Natarajan, Senthil" <senthil@pitt.edu>
To: "core-user@hadoop.apache.org" <core-user@hadoop.apache.org>
Sent: Friday, May 9, 2008 12:23:52 PM
Subject: RE: Hadoop Permission Problem

Hi Nicholas,
Here I tried as user test after I got the error (is the exception comes from slave machine?)

Exception in thread "main" org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.fs.permission.AccessControlException:
Permission denied: user=test, access=WRITE, inode="datastore":hadoop:supergroup:rwxr-xr-x

/usr/local/hadoop/bin/hadoop fs -ls
Found 1 items
/user/test/myapps    <r 11>  2674    2008-05-07 17:55        rw-r--r--       test supergroup

-----Original Message-----
From: s29752-hadoopuser@yahoo.com [mailto:s29752-hadoopuser@yahoo.com]
Sent: Friday, May 09, 2008 3:13 PM
To: core-user@hadoop.apache.org
Subject: Re: Hadoop Permission Problem

Hi Senthil,

I cannot see why it does not work.  Could you try again, do a fs -ls right after you see the
error message?

Nicholas



----- Original Message ----
From: "Natarajan, Senthil" <senthil@pitt.edu>
To: "core-user@hadoop.apache.org" <core-user@hadoop.apache.org>
Sent: Friday, May 9, 2008 11:49:49 AM
Subject: RE: Hadoop Permission Problem

Hi Nicholas,
No, I am running map/red jobs over HDFS file.
That permission is for datastore (hadoop.tmp.dir)

Here is the HDFS
/usr/local/hadoop/bin/hadoop dfs -ls /
Found 2 items
/user   <dir>           2008-05-07 17:55        rwxrwxrwx       hadoop  supergroup
/usr    <dir>           2008-05-07 17:18        rwxr-xr-x       hadoop  supergroup
[hadoop@gis14 .ssh]$ /usr/local/hadoop/bin/hadoop dfs -ls /user
Found 2 items
/user/hadoop    <dir>           2008-05-08 16:36        rwxr-xr-x       hadoop  supergroup
/user/test   <dir>           2008-05-07 17:55        rwxrwxrwx       test supergroup

Thanks,
Senthil

-----Original Message-----
From: s29752-hadoopuser@yahoo.com [mailto:s29752-hadoopuser@yahoo.com]
Sent: Friday, May 09, 2008 2:40 PM
To: core-user@hadoop.apache.org
Subject: Re: Hadoop Permission Problem

Hi Senthil,

drwxrwxrwx    4 hadoop   hadoop       4096 May  8 16:31 hadoop-hadoop
drwxrwxrwx    2 test  test      4096 May  9 09:29 hadoop-test

>From the output format, the directories above seem not HDFS directories.  Are you running
map/red jobs over local file system (e.g. Linux)?

Nicholas



----- Original Message ----
From: "Natarajan, Senthil" <senthil@pitt.edu>
To: "core-user@hadoop.apache.org" <core-user@hadoop.apache.org>
Sent: Friday, May 9, 2008 6:36:27 AM
Subject: RE: Hadoop Permission Problem

Hi Nicholas,
That's what I was wondering. Here is the datastore directory permission in the master machine.

drwxrwxrwx    5 hadoop   hadoop       4096 May  7 18:02 datastore

This datastore directory present only in the master right not on the slaves right? I couldn't
find.

After I changed the permission for datastore I restarted dfs and mapred. But still it complains
about the permission.

Even I changed all the directories in datastore to 777

drwxrwxrwx    4 hadoop   hadoop       4096 May  8 16:31 hadoop-hadoop
drwxrwxrwx    2 test  test      4096 May  9 09:29 hadoop-test

What are the places I need to change the permissions so that UserB can submit the job using
the jobtracker and tasktracker started by UserA.

Thanks,
Senthil


-----Original Message-----
From: s29752-hadoopuser@yahoo.com [mailto:s29752-hadoopuser@yahoo.com]
Sent: Thursday, May 08, 2008 8:32 PM
To: core-user@hadoop.apache.org
Subject: Re: Hadoop Permission Problem

Hi Senthil,

In the error message, it says that the permission for "datastore" is 755.  Are you sure that
you have changed it to 777?

Nicholas



----- Original Message ----
From: "Natarajan, Senthil" <senthil@pitt.edu>
To: "core-user@hadoop.apache.org" <core-user@hadoop.apache.org>
Sent: Thursday, May 8, 2008 11:57:46 AM
Subject: RE: Hadoop Permission Problem

Hi Nicholas,
Thanks it helped.

I gave permission 777 for /user
So now user "Test" can perform HDFS operations.

And also I gave permission 777 for /usr/local/hadoop/datastore on the master.

When user "Test" tries to submit the MapReduce job, getting this error

Exception in thread "main" org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.fs.permission.AccessControlException:
Permission denied: user=test, access=WRITE, inode="datastore":hadoop:supergroup:rwxr-xr-x

Where else I need to give permission so that user "Test" can submit jobs using jobtracker
and Datanode started by user "hadoop".

Thanks,
Senthil

-----Original Message-----
From: s29752-hadoopuser@yahoo.com [mailto:s29752-hadoopuser@yahoo.com]
Sent: Wednesday, May 07, 2008 5:49 PM
To: core-user@hadoop.apache.org
Subject: Re: Hadoop Permission Problem

Hi Senthil,

Since the path "myapps" is relative, copyFromLocal will copy the file to the home directory,
i.e. /user/Test/myapps in your case.  If /user/Test doesn't not exist, it will first try to
create it.  You got AccessControlException because the permission of /user is 755.

Hope this helps.

Nicholas



----- Original Message ----
From: "Natarajan, Senthil" <senthil@pitt.edu>
To: "hadoop-user@lucene.apache.org" <hadoop-user@lucene.apache.org>
Sent: Wednesday, May 7, 2008 2:36:22 PM
Subject: Hadoop Permission Problem

Hi,
My datanode and jobtracker are started by user "hadoop".
And user "Test" needs to submit the job. So if the user "Test" copies file to HDFS, there
is a permission error.
/usr/local/hadoop/bin/hadoop dfs -copyFromLocal /home/Test/somefile.txt myapps
copyFromLocal: org.apache.hadoop.fs.permission.AccessControlException: Permission denied:
user=Test, access=WRITE, inode="user":hadoop:supergroup:rwxr-xr-x
Could you please let me know how other users (other than hadoop) can access HDFS and then
submit MapReduce jobs. Where to configure or what default configuration needs to be changed.

Thanks,
Senthil

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message