Mailing-List: contact core-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-user@hadoop.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; s=serpent; d=yahoo-inc.com; c=nofws; q=dns;
	h=message-id:date:from:user-agent:mime-version:to:subject:
	references:in-reply-to:content-type:content-transfer-encoding;
	b=c4M2e5XkglpSetLe63Vwmvdt5J47nn8KdlNXtNbA7X0vmZWGiDO7S+kEwUcp+29q
Message-ID: <47C44FCA.2030100@yahoo-inc.com>
Date: Tue, 26 Feb 2008 18:43:38 +0100
From: Luca Telloli <telloli@yahoo-inc.com>
User-Agent: Thunderbird 2.0.0.9 (X11/20071115)
MIME-Version: 1.0
To: core-user@hadoop.apache.org
Subject: Re: [HOD] hdfs:///mapredsystem directory
References: <47C446FF.8030408@yahoo-inc.com>
 <4E45EC6AD219FD47AD1BC06E4EE3845DBB9A54@SNV-EXVS09.ds.corp.yahoo.com>
In-Reply-To: 
 <4E45EC6AD219FD47AD1BC06E4EE3845DBB9A54@SNV-EXVS09.ds.corp.yahoo.com>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit

Hi Mahadev,
	I'm not sure the workaround can solve the problem, because it appears 
that a subdirectory is created under that directory with the name of the 
hodring host. So if the next allocation made by a different user chooses 
the same host, the permission problem might show up again. Unless 
obviously each time the user who allocated resources deletes that 
directory during deallocation, which would be a solution for this problem.

I filed a bug: HADOOP-2899

Cheers,
Luca

Mahadev Konar wrote:
> Hi Luca,
>   This seems like a bug. The JobTracker process tries to create this
> directory if this does nto exist. And if you have two different users
> running hod clusters they will both try to create this directory and
> since only one succeeds, with permissions the directory is owned by the
> user who created the cluster first. 
> 
> A work around to this solution is to create 
> 
> Hdfs:///mapredsystem manually and make it world writable. 
> 
> Please open a bug regarding this issue.
> 
> Regards
> Mahadev
> 
>> -----Original Message-----
>> From: Luca Telloli [mailto:telloli@yahoo-inc.com]
>> Sent: Tuesday, February 26, 2008 9:06 AM
>> To: core-user@hadoop.apache.org
>> Subject: [HOD] hdfs:///mapredsystem directory
>>
>> Hello everyone,
>> 	I have a problem with directory hdfs:///mapredsystem and I'm not
>> sure
>> if it's a bug or my fault.
>>
>> Not sure if this influences what follows, but I have two users, one is
>> "hadoop" who has sudo privileges on all the nodes, the other one is
>> "luca", who has normal privileges.
>>
>> I see that each submitted job creates a hdfs:///mapredsystem
> directory,
>> created by (I guess) the hodring process. Problem is that it's not
>> cleaned up at the end of the process; for instance a use case would
> be:
>> - user hadoop allocates a cluster, the ringmaster is svr3, so a
>> /mapredsystem/svr3 directory is created
>>
>> - user hadoop deallocates the cluster, but that directory is not
> cleaned
>> up
>>
>> - user luca allocates a cluster, and the first node chosen as
> ringmaster
>> is svr3, so hodring tries to write hdfs:///mapredsystem but it fails
>>
>> - allocation succeeds, but there's no hodring running; looking at
>> 0-jobtracker/logdir/hadoop.log under the temporary directory I can
> read:
>> 2008-02-26 17:28:42,567 WARN org.apache.hadoop.mapred.JobTracker:
> Error
>> starting tracker: org.apache.hadoop.ipc.RemoteException:
>> org.apache.hadoop.fs.permission.AccessControlException: Permission
>> denied: user=luca, access=WRITE,
>> inode="mapredsystem":hadoop:supergroup:rwxr-xr-x
>>
>> Am I doing anything wrong?
>> Cheers,
>> Luca
>