hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From maisnam ns <maisnam...@gmail.com>
Subject Re: Basic Doubt in Hadoop
Date Wed, 17 Apr 2013 09:06:38 GMT
@Bejoy
Adding a little bit here,the ouput of map task writes first to a memory
buffer, and when contents reaches a threshold a background thread will
write the contents to disk.

Niranjan Singh


On Wed, Apr 17, 2013 at 1:06 PM, Ramesh R Nair <rameshrkcoder@gmail.com>wrote:

> Hi Bejoy,
>
>    Regarding the output of Map phase, does  Hadoop store it in local fs or
> in HDFS.
>    I believe it is in the former. Correct me if I am wrong.
>
> Regards
> Ramesh
>
>
> On Wed, Apr 17, 2013 at 10:30 AM, <bejoy.hadoop@gmail.com> wrote:
>
>> The data is in HDFS in case of WordCount MR sample.
>>
>> In hdfs, you have the metadata in NameNode and actual data as blocks
>> replicated across DataNodes.
>>
>> In case of reducer, If a reducer is running on a particular node then you
>> have one replica of the blocks in the same node (If there is no space
>> issues) and rest replicas on other nodes.
>> Regards
>> Bejoy KS
>>
>> Sent from remote device, Please excuse typos
>> ------------------------------
>> *From: * Raj Hadoop <hadoopraj@yahoo.com>
>> *Date: *Tue, 16 Apr 2013 21:49:34 -0700 (PDT)
>> *To: *user@hadoop.apache.org<user@hadoop.apache.org>
>> *ReplyTo: * user@hadoop.apache.org
>> *Subject: *Basic Doubt in Hadoop
>>
>> Hi,
>>
>> I am new to Hadoop. I started reading the standard Wordcount program. I
>> got this basic doubt in Hadoop.
>>
>> After the Map - Reduce is done, where is the output generated?  Does the
>> reducer ouput sit on individual DataNodes ? Please advise.
>>
>>
>> Thanks,
>> Raj
>>
>
>

Mime
View raw message