hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hemanth Yamijala <yhema...@yahoo-inc.com>
Subject Re: [HOD] hdfs:///mapredsystem directory
Date Wed, 27 Feb 2008 10:39:39 GMT
Mahadev Konar wrote:
> Hi Luca,
>   Can you do a ls -l on /mapredsystem and send the output? According to
> permissions for mapreduce the system directories created by jobtracker
> should be world writable so permissions should have worked as it is for
> hod.
>
>   
No, it doesn't appear to be working that way. The mapred system 
directory specified by HOD for a jobtracker is /mapredsystem/<hostname>. 
When the job tracker starts, it creates this directory with permissions 
rwx-wx-wx. It does not clean it up when it stops. But when it is started 
again on the same node, I think (may be wrong here) it tries to clean up 
the directory (possibly to handle crashes etc ?). This seems equivalent 
to issuing a "hadoop dfs -rmr /mapredsystem/<hostname>"which in turn rm 
-r of unix. So, it basically has to read the directory to recursively 
delete; and because there are no read permissions, it fails.

A simple experiment to simulate the same set of DFS operations using 2 
different users, without using mapred, give the same result. Enabling 
read permissions allows the delete to work.

There appear several ways we can fix this, and that discussion should go 
on HADOOP-2899. Will post my comments there.

Thanks
hemanth

Mime
View raw message