hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From nagarjuna kanamarlapudi <nagarjuna.kanamarlap...@gmail.com>
Subject Re: Understanding MapReduce source code : Flush operations
Date Mon, 06 Jan 2014 19:07:10 GMT
I want to have a look at the code where of flush operations that happens
after the reduce phase.

Reducer writes the output to OutputFormat which inturn pushes that to
memory and once it reaches 90% of chunk size it starts to flush the reducer
output.

I essentially want to look at the code of that flushing operation.


What is the class(es) I need to look into


On Mon, Jan 6, 2014 at 11:23 PM, Hardik Pandya <smarty.juice@gmail.com>wrote:

> Please do not tell me since last 2.5 years you have not used virtual
> Hadoop environment to debug your Map Reduce application before deploying to
> Production environment
>
> No one can stop you looking at the code , Hadoop and its ecosystem is
> open-source
>
>
> On Mon, Jan 6, 2014 at 9:35 AM, nagarjuna kanamarlapudi <
> nagarjuna.kanamarlapudi@gmail.com> wrote:
>
>>
>>
>> ---------- Forwarded message ----------
>> From: nagarjuna kanamarlapudi <nagarjuna.kanamarlapudi@gmail.com>
>>  Date: Mon, Jan 6, 2014 at 6:39 PM
>> Subject: Understanding MapReduce source code : Flush operations
>> To: mapreduce-user@hadoop.apache.org
>>
>>
>>  Hi,
>>
>> I am using hadoop/ map reduce for aout 2.5 years. I want to understand
>> the internals of the hadoop source code.
>>
>> Let me put my requirement very clear.
>>
>> I want to have a look at the code where of flush operations that happens
>> after the reduce phase.
>>
>> Reducer writes the output to OutputFormat which inturn pushes that to
>> memory and once it reaches 90% of chunk size it starts to flush the reducer
>> output.
>>
>> I essentially want to look at the code of that flushing operation.
>>
>>
>>
>>
>> Regards,
>> Nagarjuna K
>>
>>
>

Mime
View raw message