hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj Das" <d...@yahoo-inc.com>
Subject RE: Hadoop 0.15.0 - Reporter issue w/ timing out
Date Sat, 10 Nov 2007 14:35:53 GMT
There has been a change with respect to the way since progress reporting is
done since 0.14. The application has to explicitly send the status
(incrCounter doesn't send any status). Even if the application hasn't made
any progress, it is okay to call setStatus with the earlier status.

> -----Original Message-----
> From: Derek Gottfrid [mailto:dgottfrid@gmail.com] 
> Sent: Saturday, November 10, 2007 2:09 AM
> To: hadoop-user@lucene.apache.org
> Subject: Hadoop 0.15.0 - Reporter issue w/ timing out
> I have a very simple map task  that gets a filename - reads 
> and decompresss that file and stores the decompressed file away.
> As it is reading and writing it does Report.incrCounter w/ 
> the number of bytes decompressed and written. It works and I 
> can see the incrementer chugging along but then even though 
> the process is still calling incrCounter - it stops and 
> basically the task gets timed out and killed. This use to 
> work but doesn't in 0.15.0 - interestingly if I add a 
> report.setStatus() in there it stays alive and doesn't get killed.
> Is this a bug? Am I doing something wrong?
> thanks,
> derek

View raw message