hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shahnawaz Saifi <shahsa...@gmail.com>
Subject Re: problem regarding the hadoop
Date Fri, 01 Jul 2011 08:59:56 GMT
Harsh,

Thanks for sharing. Recently, I ran into multiple user issue on cluster
because all the time we were using same user to run MRs so I found below
work around for it.

I am curious to know, How do we separate HDFS processes for users?



On Fri, Jul 1, 2011 at 1:39 PM, Harsh J <harsh@cloudera.com> wrote:

> Shahnawaz,
>
> If required, the MR and the HDFS processes can be user separated too.
> Its generally a good thing to do in practice (so that MR daemons and
> user don't get "superuser" access to HDFS files).
>
> FWIW, I usually flip open the cluster setup guide on ccp.cloudera.com
> and everything comes up right when I follow that (its got footnotes
> too, yay!).
>
> But yes, managing local permissions is quite a pain to do manually,
> and I believe the new bigtop incubator entrant is gonna help in
> setting up clusters painlessly as we go forward.
>
> On Fri, Jul 1, 2011 at 12:35 PM, Shahnawaz Saifi <shahsaifi@gmail.com>
> wrote:
> > You can create multiple users on Cluster because it follows LSM (Linux
> > security model). But Hadoop related process NN, DN, JT, TT need to be
> > started with one user e.g. hadoop.
> >
> > Thanks,
> > Shah
> >
> > On Fri, Jul 1, 2011 at 4:57 AM, Mitra Kaseebhotla <
> > mitra.kaseebhotla@gmail.com> wrote:
> >
> >> Not to divert the question here. I would like to know how we should
> manage
> >> the cluster users (I am a newbie)? insead of having one user(hadoop)
> creds
> >> shared?
> >>
> >> Thanks
> >> Mitra.
> >>
> >> On Thu, Jun 30, 2011 at 4:23 PM, Paul Rimba <paul.rimba@gmail.com>
> wrote:
> >>
> >> > sudo chown -R hadoop:hadoop /usr/local/hadoop.
> >> > That will give the directory ownership over to your hadoop account.
> >> >
> >> > On Fri, Jul 1, 2011 at 5:07 AM, Dhruv Kumar <dkumar@ecs.umass.edu>
> >> wrote:
> >> >
> >> > > It is a permission issue. Are you sure that the account "hadoop" has
> >> read
> >> > > and write access to /usr/local/* directories?
> >> > >
> >> > > The installation of Hadoop has always been effortless for me. Just
> >> follow
> >> > > the instructions step by step given on:
> >> > > http://hadoop.apache.org/common/docs/stable/single_node_setup.html
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > On Thu, Jun 30, 2011 at 11:08 AM, Ashish Tamrakar
> >> > > <ashishtamrakar@gmail.com>wrote:
> >> > >
> >> > > > I am having a problem starting my hadoop. i setup my for my
> >> single-node
> >> > > > cluster. please help me to solve this out!!!!
> >> > > >
> >> > > > hadoop@ashishpc:~$ /usr/local/hadoop/bin/start-all.sh
> >> > > > starting namenode, logging to
> >> > > > /usr/local/hadoop/bin/../logs/hadoop-hadoop-namenode-ashishpc.out
> >> > > > /usr/local/hadoop/bin/hadoop-daemon.sh: line 117:
> >> > > > /usr/local/hadoop/bin/../logs/hadoop-hadoop-namenode-ashishpc.out:
> >> > > > Permission denied
> >> > > > head: cannot open
> >> > > >
> `/usr/local/hadoop/bin/../logs/hadoop-hadoop-namenode-ashishpc.out'
> >> for
> >> > > > reading: No such file or directory
> >> > > > localhost: starting datanode, logging to
> >> > > > /usr/local/hadoop/bin/../logs/hadoop-hadoop-datanode-ashishpc.out
> >> > > > localhost: /usr/local/hadoop/bin/hadoop-daemon.sh: line 117:
> >> > > > /usr/local/hadoop/bin/../logs/hadoop-hadoop-datanode-ashishpc.out:
> >> > > > Permission denied
> >> > > > localhost: head: cannot open
> >> > > >
> `/usr/local/hadoop/bin/../logs/hadoop-hadoop-datanode-ashishpc.out'
> >> for
> >> > > > reading: No such file or directory
> >> > > > localhost: starting secondarynamenode, logging to
> >> > > >
> >> > >
> >> >
> >>
> /usr/local/hadoop/bin/../logs/hadoop-hadoop-secondarynamenode-ashishpc.out
> >> > > > localhost: /usr/local/hadoop/bin/hadoop-daemon.sh: line 117:
> >> > > >
> >> > >
> >> >
> >>
> /usr/local/hadoop/bin/../logs/hadoop-hadoop-secondarynamenode-ashishpc.out:
> >> > > > Permission denied
> >> > > > localhost: head: cannot open
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> `/usr/local/hadoop/bin/../logs/hadoop-hadoop-secondarynamenode-ashishpc.out'
> >> > > > for reading: No such file or directory
> >> > > > starting jobtracker, logging to
> >> > > >
> /usr/local/hadoop/bin/../logs/hadoop-hadoop-jobtracker-ashishpc.out
> >> > > > /usr/local/hadoop/bin/hadoop-daemon.sh: line 117:
> >> > > >
> /usr/local/hadoop/bin/../logs/hadoop-hadoop-jobtracker-ashishpc.out:
> >> > > > Permission denied
> >> > > > head: cannot open
> >> > > >
> `/usr/local/hadoop/bin/../logs/hadoop-hadoop-jobtracker-ashishpc.out'
> >> > for
> >> > > > reading: No such file or directory
> >> > > > localhost: starting tasktracker, logging to
> >> > > >
> /usr/local/hadoop/bin/../logs/hadoop-hadoop-tasktracker-ashishpc.out
> >> > > > localhost: /usr/local/hadoop/bin/hadoop-daemon.sh: line 117:
> >> > > >
> /usr/local/hadoop/bin/../logs/hadoop-hadoop-tasktracker-ashishpc.out:
> >> > > > Permission denied
> >> > > > localhost: head: cannot open
> >> > > >
> >> `/usr/local/hadoop/bin/../logs/hadoop-hadoop-tasktracker-ashishpc.out'
> >> > > for
> >> > > > reading: No such file or directory
> >> > > >
> >> > >
> >> >
> >>
> >>
> >>
> >> --
> >>  ________________________________________
> >>
> >>  To Understand Recursion,
> >>  You must first Understand Recursion
> >>  ________________________________________
> >>
> >
>
>
>
> --
> Harsh J
>



-- 
Thanks,
Shah

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message