hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun C Murthy <...@yahoo-inc.com>
Subject Re: Difference between local mode and distributed mode
Date Thu, 06 Mar 2008 16:27:52 GMT

On Mar 6, 2008, at 5:23 AM, Naama Kraus wrote:

> Thanks, that's fine with me then. Naama
>

Please note that you can also run hadoop in a 'psuedo-distributed'  
mode on a single node where you can mimic all characteristics (all?  
except, err... performance of course!) of a large cluster:

http://hadoop.apache.org/core/docs/r0.16.0/ 
quickstart.html#SingleNodeSetup

Arun

> On Thu, Mar 6, 2008 at 3:21 PM, Enis Soztutar  
> <enis.soz.nutch@gmail.com>
> wrote:
>
>> Hi,
>>
>> LocalJobRunner uses just 0 or 1 reduce. This is because running in  
>> local
>> mode is only supported for testing purposes.
>> Although you can simulate distribute mode in local, by using
>> MiniMRCluster and MiniDFSCluster under src/test.
>>
>> Best wishes
>> Enis
>>
>> Naama Kraus wrote:
>>> Hi,
>>>
>>> I ran a simple MapReduce job which defines 3 reducers:
>>>
>>> *conf.setNumReduceTasks(3);*
>>>
>>> When running on top of HDFS (distributed mode), I got 3 out files  
>>> as I
>>> expected.
>>> When running on top of a local files system (local mode), I got 1  
>>> file
>> and
>>> not 3.
>>>
>>> My question is whether the behavior in the local is the expected  
>>> one, is
>> a
>>> bug, or maybe something needs to be configured in the local mode in
>> order to
>>> get the 3 files as well ?
>>>
>>> Thanks for an insight, Naama
>>>
>>>
>>
>
>
>
> -- 
> oo 00 oo 00 oo 00 oo 00 oo 00 oo 00 oo 00 oo 00 oo 00 oo 00 oo 00  
> oo 00 oo
> 00 oo 00 oo
> "If you want your children to be intelligent, read them fairy  
> tales. If you
> want them to be more intelligent, read them more fairy tales." (Albert
> Einstein)


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message