hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vishal Shah" <vish...@rediff.co.in>
Subject RE: Configuration and Hadoop cluster setup
Date Fri, 25 May 2007 07:23:33 GMT
Hi Avinash,

  Can you share your hadoop-site.xml, mapred-default.xml and slaves files?
Most probably, you have not set the jobtracker properly in the
hadoop-site.xml conf file. Check the property mapred.job.tracker property in
your file. It should look something like this:

<property>
  <name>mapred.job.tracker</name>
  <value>fully.qualified.domainname:40000</value>
  <description>The host and port that the MapReduce job tracker runs
  at.  If "local", then jobs are run in-process as a single map
  and reduce task.
  </description>
</property>

-vishal.

-----Original Message-----
From: Mahadev Konar [mailto:mahadev@yahoo-inc.com] 
Sent: Friday, May 25, 2007 5:54 AM
To: hadoop-user@lucene.apache.org
Subject: RE: Configuration and Hadoop cluster setup

Hi,
  When you run the job, you need to set the environment variable
HADOOP_CONF_DIR to the configuration directory that has the configuration
file pointing to the right jobtracker.

Regards
Mahadev

> -----Original Message-----
> From: Phantom [mailto:ghostwhoowalks@gmail.com]
> Sent: Thursday, May 24, 2007 4:51 PM
> To: hadoop-user@lucene.apache.org
> Subject: Re: Configuration and Hadoop cluster setup
> 
> Yes the files are the same and I am starting the tasks on the namenode
> server. I also figured what my problem was with respect to not being able
> to
> start the namenode and job tracker on the same machine. I had to reformat
> the file system. But the all this still doesn't cause the WordCount sample
> to run in a distributed fashion. I can tell this because the
> LocalJobRunner
> is being used. Do I need to specify the config file to the running
> instance
> of the program ? If so how do I do that ?
> 
> Thanks
> A
> 
> On 5/24/07, Dennis Kubes <nutch-dev@dragonflymc.com> wrote:
> >
> >
> >
> > Phantom wrote:
> > > I am trying to run Hadoop on a cluster of 3 nodes. The namenode and
> the
> > > jobtracker web UI work. I have the namenode running on node A and job
> > > tracker running on node B. Is it true that namenode and jobtracker
> > cannot
> > > run on the same box ?
> >
> > The namenode and the jobtracker can most definitely run on the same box.
> >   As far as I know this is the preferred configuration.
> >
> > Also if I want to run the examples on the cluster is
> > > there anything special that needs to be done. When I run the example
> > > WordCount on a machine C (which is a task tracker and not a job
> tracker)
> > > the
> > > LocalJobRunner is invoked all the time. I am guessing this means that
> > the
> > > map tasks are running locally. How can I distribute this on the
> cluster
> > ?
> > > Please advice.
> >
> > Are the conf files on machine C the same as the namenode/jobtracker?
> > Are they pointing to the namenode and jobtracker or are they pointing to
> > local in the hadoop-site.xml file.  Also we have found it easier
> > (although not necessarily better) to start tasks on the namenode server.
> >
> > It would be helpful to have more information about what is happening and
> > your setup as that would help myself and others on the list debug what
> > may be occurring.
> >
> > Dennis Kubes
> >
> > >
> > > Thanks
> > > Avinash
> > >
> >


Mime
View raw message