hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrzej Bialecki ...@getopt.org>
Subject Lack of progress info in map tasks
Date Thu, 08 Mar 2007 22:37:38 GMT
Hi all,

Is it just me, or is there something strange with Hadoop since ~0.10 or 
thereabout .. With older version of Hadoop I would get a nice often 
updated progress status for each map task. What I'm seeing now is that 
map tasks stay at 0.0% and then finally jump to 100.0% and finish. 
Consequently, for jobs with small number of long-running map tasks, the 
progress update is very coarse.

As I understand, this progress meter (in absence of map tasks explicitly 
setting the progress) was based on the RecordReader reporting of how 
much of the current split has been read. Is this something that got 
broken on the way? If not, what's the reason for this, and how to fix it?

Best regards,
Andrzej Bialecki     <><
 ___. ___ ___ ___ _ _   __________________________________
[__ || __|__/|__||\/|  Information Retrieval, Semantic Web
___|||__||  \|  ||  |  Embedded Unix, System Integration
http://www.sigram.com  Contact: info at sigram dot com

View raw message