hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mafish Liu" <maf...@gmail.com>
Subject Re: dynamically adding slaves to hadoop cluster
Date Mon, 10 Mar 2008 01:52:14 GMT
On Mon, Mar 10, 2008 at 9:47 AM, Mafish Liu <mafish@gmail.com> wrote:

> You should do the following steps:
> 1. Have hadoop deployed on the new node with the same directory structure
> and configuration.
> 2. Just run $HADOOP_HOME/bin/hadoop datanode and jobtracker.

Addition: do not run "bin/hadoop namenode -format" before you run datanode,
or you will get a error like "Incompatible namespaceIDs ..."

>
>
> Datanode and jobtracker will contact to namenode specified in hadoop
> configuration file automatically and finish adding new node to the hadoop
> cluster.
>
>
> On Mon, Mar 10, 2008 at 4:56 AM, Aaron Kimball <ak@cs.washington.edu>
> wrote:
>
> > Yes. You should have the same hadoop-site across all your slaves. They
> > will need to know the DNS name for the namenode and jobtracker.
> >
> > - Aaron
> >
> > tjohn wrote:
> > >
> > > Mahadev Konar wrote:
> > >
> > >> I believe (as far as I remember) you should be able to add the node
> > by
> > >> bringing up the datanode or tasktracker on the remote machine. The
> > >> Namenode or the jobtracker (I think) does not check for the nodes in
> > the
> > >> slaves file. The slaves file is just to start up all the daemon's by
> > >> ssshing to all the nodes in the slaves file during startup. So you
> > >> should just be able to startup the datanode pointing to correct
> > namenode
> > >> and it should work.
> > >>
> > >> Regards
> > >> Mahadev
> > >>
> > >>
> > >>
> > >
> > > Sorry for my ignorance... To make a datanode/tasktraker point to the
> > > namenode what should i do? Have i to edit the hadoop-site.xml? Thanks
> > >
> > > John
> > >
> > >
> >
>
>
>
> --
> Mafish@gmail.com
> Institute of Computing Technology, Chinese Academy of Sciences, Beijing.
>



-- 
Mafish@gmail.com
Institute of Computing Technology, Chinese Academy of Sciences, Beijing.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message