hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jim the Standing Bear" <standingb...@gmail.com>
Subject Re: clarification on running a job locally
Date Thu, 01 Nov 2007 20:37:40 GMT
Hi Ted,

It is funny how I am having so much difficulties with hadoop... since
i am on this subject, let me ask another stupid question:

In order to run a hadoop job, I must zip it into a jar, and then use
$HADOOP_HOME/bin/hadoop to launch it, correct?  I cannot simply use
"java" to directly launch it, right?

best wishes

Yiming


On 11/1/07, Ted Dunning <tdunning@veoh.com> wrote:
>
> When running a job standalone (typically for debugging), it suffices for me
> to simply add -Dmapred.job.tracker=local to the program command line.
>
> If you simply want to run the program on a single node, then you can just
> stop the other node.  Using local storage with distributed programs is not
> recommended.
>
>
> On 11/1/07 12:35 PM, "Jim the Standing Bear" <standingbear@gmail.com> wrote:
>
> > Hello,
> >
> > I am in need of some clarifications on how to run a hadoop job locally.
> >
> > The cluster was originally set up to have two nodes, where one of them
> > also acts as the master node and job tracker.
> >
> > According to the wiki, I can run a job locally by altering
> > "mapred.job.tracker" and "fs.default.name" properties to "local" in
> > hadoop-site.xml.  But when I start the server, it stack dumped:
> >
> > localhost: starting secondarynamenode, logging to /home/blahblahblah
> > localhost: Exception in thread "main" java.lang.RuntimeException: Not
> > a host:port pair: local
> >
> > Apparently it didn't like the value "local"?
> >
> > Also, the wiki noted that all these XML configuration files should be
> > included somewhere in the class path to the job, does it mean I need
> > to include the XMLs as I do jars?
> >
> > Thank
> >
> > -- Jim
>
>


-- 
--------------------------------------
Standing Bear Has Spoken
--------------------------------------

Mime
View raw message