hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris K Wensel <ch...@wensel.net>
Subject Re: clarification on running a job locally
Date Thu, 01 Nov 2007 20:23:58 GMT
hmm.. maybe we are talking about different things. i'll politely pass  
this one on.


On Nov 1, 2007, at 1:05 PM, Jim the Standing Bear wrote:

> Hi Chris,
>
> I meant "Local Runner".
>
> From the sounds of your email, it seems that modifying the two
> fore-mentioned properties is not enough to get a cluster node to run
> as a local runner?
>
> Also, what about "including .xml to the class path"?  Do I include
> them like jar files?  My ant script has included all the external jars
> in the lib/ directory - do I need to do the same for the xml files?
>
> thanks
>
> -- Jim
>
> On 11/1/07, Chris K Wensel <chris@wensel.net> wrote:
>> Hey Jim
>>
>> what do you mean by local?
>>
>> This wiki page defines two 'local' modes.
>> http://wiki.apache.org/lucene-hadoop/QuickStart
>>
>> One with a local cluster, and one with all jobs running standalone.
>>
>> The conf dir that ships with Hadoop is setup for standalone.
>>
>> I find it useful to have a copy of 'conf' named 'conf-localhost' that
>> has the local cluster version of 'hadoop-site.xml'. then use
>>
>>> ./bin/hadoop --config ./conf-localhost ....
>>
>> to start her up.. (after following the other wiki steps)
>>
>> cheers
>> ckw
>>
>> On Nov 1, 2007, at 12:35 PM, Jim the Standing Bear wrote:
>>
>>> Hello,
>>>
>>> I am in need of some clarifications on how to run a hadoop job
>>> locally.
>>>
>>> The cluster was originally set up to have two nodes, where one of  
>>> them
>>> also acts as the master node and job tracker.
>>>
>>> According to the wiki, I can run a job locally by altering
>>> "mapred.job.tracker" and "fs.default.name" properties to "local" in
>>> hadoop-site.xml.  But when I start the server, it stack dumped:
>>>
>>> localhost: starting secondarynamenode, logging to /home/blahblahblah
>>> localhost: Exception in thread "main" java.lang.RuntimeException:  
>>> Not
>>> a host:port pair: local
>>>
>>> Apparently it didn't like the value "local"?
>>>
>>> Also, the wiki noted that all these XML configuration files  
>>> should be
>>> included somewhere in the class path to the job, does it mean I need
>>> to include the XMLs as I do jars?
>>>
>>> Thank
>>>
>>> -- Jim
>>
>> Chris K Wensel
>> chris@wensel.net
>> http://chris.wensel.net/
>>
>>
>>
>>
>
>
> -- 
> --------------------------------------
> Standing Bear Has Spoken
> --------------------------------------

Chris K Wensel
chris@wensel.net
http://chris.wensel.net/




Mime
View raw message