hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mahadev Konar" <maha...@yahoo-inc.com>
Subject RE: [HOD] hdfs:///mapredsystem directory
Date Tue, 26 Feb 2008 18:22:09 GMT
Hi Luca,
  Can you do a ls -l on /mapredsystem and send the output? According to
permissions for mapreduce the system directories created by jobtracker
should be world writable so permissions should have worked as it is for
hod.


Regards
Mahadev

> -----Original Message-----
> From: Luca Telloli [mailto:telloli@yahoo-inc.com]
> Sent: Tuesday, February 26, 2008 9:44 AM
> To: core-user@hadoop.apache.org
> Subject: Re: [HOD] hdfs:///mapredsystem directory
> 
> Hi Mahadev,
> 	I'm not sure the workaround can solve the problem, because it
> appears
> that a subdirectory is created under that directory with the name of
the
> hodring host. So if the next allocation made by a different user
chooses
> the same host, the permission problem might show up again. Unless
> obviously each time the user who allocated resources deletes that
> directory during deallocation, which would be a solution for this
problem.
> 
> I filed a bug: HADOOP-2899
> 
> Cheers,
> Luca
> 
> Mahadev Konar wrote:
> > Hi Luca,
> >   This seems like a bug. The JobTracker process tries to create this
> > directory if this does nto exist. And if you have two different
users
> > running hod clusters they will both try to create this directory and
> > since only one succeeds, with permissions the directory is owned by
the
> > user who created the cluster first.
> >
> > A work around to this solution is to create
> >
> > Hdfs:///mapredsystem manually and make it world writable.
> >
> > Please open a bug regarding this issue.
> >
> > Regards
> > Mahadev
> >
> >> -----Original Message-----
> >> From: Luca Telloli [mailto:telloli@yahoo-inc.com]
> >> Sent: Tuesday, February 26, 2008 9:06 AM
> >> To: core-user@hadoop.apache.org
> >> Subject: [HOD] hdfs:///mapredsystem directory
> >>
> >> Hello everyone,
> >> 	I have a problem with directory hdfs:///mapredsystem and I'm not
> >> sure
> >> if it's a bug or my fault.
> >>
> >> Not sure if this influences what follows, but I have two users, one
is
> >> "hadoop" who has sudo privileges on all the nodes, the other one is
> >> "luca", who has normal privileges.
> >>
> >> I see that each submitted job creates a hdfs:///mapredsystem
> > directory,
> >> created by (I guess) the hodring process. Problem is that it's not
> >> cleaned up at the end of the process; for instance a use case would
> > be:
> >> - user hadoop allocates a cluster, the ringmaster is svr3, so a
> >> /mapredsystem/svr3 directory is created
> >>
> >> - user hadoop deallocates the cluster, but that directory is not
> > cleaned
> >> up
> >>
> >> - user luca allocates a cluster, and the first node chosen as
> > ringmaster
> >> is svr3, so hodring tries to write hdfs:///mapredsystem but it
fails
> >>
> >> - allocation succeeds, but there's no hodring running; looking at
> >> 0-jobtracker/logdir/hadoop.log under the temporary directory I can
> > read:
> >> 2008-02-26 17:28:42,567 WARN org.apache.hadoop.mapred.JobTracker:
> > Error
> >> starting tracker: org.apache.hadoop.ipc.RemoteException:
> >> org.apache.hadoop.fs.permission.AccessControlException: Permission
> >> denied: user=luca, access=WRITE,
> >> inode="mapredsystem":hadoop:supergroup:rwxr-xr-x
> >>
> >> Am I doing anything wrong?
> >> Cheers,
> >> Luca
> >


Mime
View raw message