hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saptarshi Guha <saptarshi.g...@gmail.com>
Subject Re: When is configure and close run
Date Wed, 24 Jun 2009 17:00:00 GMT
Thank you! Just to confirm. Consider a JVM (that is being reused), has
to reduce K1,{V11,V12,V13..} and K2,{V21,V22,V23,....}. Then the
configure and close methods are called once each for both K1,{V11,...}
and K2,{V2,....}?

Is my understanding correct?

Once again, there is no combiner, and it makes sense that it is not called.

Thank you
Saptarshi


On Mon, Jun 22, 2009 at 10:55 PM, jason hadoop<jason.hadoop@gmail.com> wrote:
> configure and close are run for each task, mapper and reducer. The configure
> and close are NOT run on the combiner class.
>
> On Mon, Jun 22, 2009 at 9:23 AM, Saptarshi Guha <saptarshi.guha@gmail.com>
> wrote:
>>
>> Hello,
>> In a mapreduce job, a given map JVM will run N map tasks. Are the
>> configure and close methods executed for every one of these N tasks?
>> Or is configure executed once when the JVM starts and the close method
>> executed once when all N have been completed?
>>
>> I have the same question for the reduce task. Will it be run before
>> for every reduce task? And close is run when all the values for a
>> given key have been processed?
>>
>> We can assume there isn't a combiner.
>>
>> Regards
>> Saptarshi
>
>
>
> --
> Pro Hadoop, a book to guide you from beginner to hadoop mastery,
> http://www.amazon.com/dp/1430219424?tag=jewlerymall
> www.prohadoopbook.com a community for Hadoop Professionals
>

Mime
View raw message