hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jim the Standing Bear" <standingb...@gmail.com>
Subject Re: clarification on running a job locally
Date Thu, 01 Nov 2007 20:05:53 GMT
Hi Chris,

I meant "Local Runner".

>From the sounds of your email, it seems that modifying the two
fore-mentioned properties is not enough to get a cluster node to run
as a local runner?

Also, what about "including .xml to the class path"?  Do I include
them like jar files?  My ant script has included all the external jars
in the lib/ directory - do I need to do the same for the xml files?

thanks

-- Jim

On 11/1/07, Chris K Wensel <chris@wensel.net> wrote:
> Hey Jim
>
> what do you mean by local?
>
> This wiki page defines two 'local' modes.
> http://wiki.apache.org/lucene-hadoop/QuickStart
>
> One with a local cluster, and one with all jobs running standalone.
>
> The conf dir that ships with Hadoop is setup for standalone.
>
> I find it useful to have a copy of 'conf' named 'conf-localhost' that
> has the local cluster version of 'hadoop-site.xml'. then use
>
>  > ./bin/hadoop --config ./conf-localhost ....
>
> to start her up.. (after following the other wiki steps)
>
> cheers
> ckw
>
> On Nov 1, 2007, at 12:35 PM, Jim the Standing Bear wrote:
>
> > Hello,
> >
> > I am in need of some clarifications on how to run a hadoop job
> > locally.
> >
> > The cluster was originally set up to have two nodes, where one of them
> > also acts as the master node and job tracker.
> >
> > According to the wiki, I can run a job locally by altering
> > "mapred.job.tracker" and "fs.default.name" properties to "local" in
> > hadoop-site.xml.  But when I start the server, it stack dumped:
> >
> > localhost: starting secondarynamenode, logging to /home/blahblahblah
> > localhost: Exception in thread "main" java.lang.RuntimeException: Not
> > a host:port pair: local
> >
> > Apparently it didn't like the value "local"?
> >
> > Also, the wiki noted that all these XML configuration files should be
> > included somewhere in the class path to the job, does it mean I need
> > to include the XMLs as I do jars?
> >
> > Thank
> >
> > -- Jim
>
> Chris K Wensel
> chris@wensel.net
> http://chris.wensel.net/
>
>
>
>


-- 
--------------------------------------
Standing Bear Has Spoken
--------------------------------------

Mime
View raw message