hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Owen O'Malley <o...@yahoo-inc.com>
Subject Re: Re-reduce, without re-map
Date Wed, 04 Apr 2007 14:39:40 GMT

On Apr 4, 2007, at 2:57 AM, Mathijs Homminga wrote:

>> Your reduce task may fail because there are too many values  
>> associated with
>> some key and it takes more than 10 minutes to process the key.  
>> Please try to
>> let your reduce task explicitly notify the task tracker that "I am  
>> alive" by
>> doing report.setStatus(String) once, for example, every 100 or  
>> 1000 values.

By the way, a fix for this problem just went in yesterday:
https://issues.apache.org/jira/browse/HADOOP-1105

Furthermore, if all you want to do is say "I'm alive", it is better  
to use:
report.progress().

-- Owen

Mime
View raw message