hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Luca Telloli <tell...@yahoo-inc.com>
Subject Re: [HOD] hdfs:///mapredsystem directory
Date Tue, 26 Feb 2008 17:43:38 GMT
Hi Mahadev,
	I'm not sure the workaround can solve the problem, because it appears 
that a subdirectory is created under that directory with the name of the 
hodring host. So if the next allocation made by a different user chooses 
the same host, the permission problem might show up again. Unless 
obviously each time the user who allocated resources deletes that 
directory during deallocation, which would be a solution for this problem.

I filed a bug: HADOOP-2899

Cheers,
Luca

Mahadev Konar wrote:
> Hi Luca,
>   This seems like a bug. The JobTracker process tries to create this
> directory if this does nto exist. And if you have two different users
> running hod clusters they will both try to create this directory and
> since only one succeeds, with permissions the directory is owned by the
> user who created the cluster first. 
> 
> A work around to this solution is to create 
> 
> Hdfs:///mapredsystem manually and make it world writable. 
> 
> Please open a bug regarding this issue.
> 
> Regards
> Mahadev
> 
>> -----Original Message-----
>> From: Luca Telloli [mailto:telloli@yahoo-inc.com]
>> Sent: Tuesday, February 26, 2008 9:06 AM
>> To: core-user@hadoop.apache.org
>> Subject: [HOD] hdfs:///mapredsystem directory
>>
>> Hello everyone,
>> 	I have a problem with directory hdfs:///mapredsystem and I'm not
>> sure
>> if it's a bug or my fault.
>>
>> Not sure if this influences what follows, but I have two users, one is
>> "hadoop" who has sudo privileges on all the nodes, the other one is
>> "luca", who has normal privileges.
>>
>> I see that each submitted job creates a hdfs:///mapredsystem
> directory,
>> created by (I guess) the hodring process. Problem is that it's not
>> cleaned up at the end of the process; for instance a use case would
> be:
>> - user hadoop allocates a cluster, the ringmaster is svr3, so a
>> /mapredsystem/svr3 directory is created
>>
>> - user hadoop deallocates the cluster, but that directory is not
> cleaned
>> up
>>
>> - user luca allocates a cluster, and the first node chosen as
> ringmaster
>> is svr3, so hodring tries to write hdfs:///mapredsystem but it fails
>>
>> - allocation succeeds, but there's no hodring running; looking at
>> 0-jobtracker/logdir/hadoop.log under the temporary directory I can
> read:
>> 2008-02-26 17:28:42,567 WARN org.apache.hadoop.mapred.JobTracker:
> Error
>> starting tracker: org.apache.hadoop.ipc.RemoteException:
>> org.apache.hadoop.fs.permission.AccessControlException: Permission
>> denied: user=luca, access=WRITE,
>> inode="mapredsystem":hadoop:supergroup:rwxr-xr-x
>>
>> Am I doing anything wrong?
>> Cheers,
>> Luca
> 


Mime
View raw message