hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amogh Vasekar <am...@yahoo-inc.com>
Subject RE: Why is single reducer called twice?
Date Tue, 28 Jul 2009 05:24:55 GMT

>> the reducer is called a
>>second time to do nothing, before all is done

Can you elaborate please?

Amogh
-----Original Message-----
From: Mark Kerzner [mailto:markkerzner@gmail.com] 
Sent: Monday, July 27, 2009 8:51 PM
To: core-user@hadoop.apache.org
Subject: Why is single reducer called twice?

Hi,
In my code, I set the number of reducers to 1,

conf.setNumReduceTasks(1);

and it all works as intended, but I notice that the reducer is called a
second time to do nothing, before all is done. Can I use this side effect to
close the static resources (like a zip output stream) that I open on the
first call?

Thank you,
Mark

Mime
View raw message