hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrzej Bialecki ...@getopt.org>
Subject Re: Lack of progress info in map tasks
Date Thu, 15 Mar 2007 21:36:20 GMT
Andrzej Bialecki wrote:
> Hi all,
>
> Is it just me, or is there something strange with Hadoop since ~0.10 
> or thereabout .. With older version of Hadoop I would get a nice often 
> updated progress status for each map task. What I'm seeing now is that 
> map tasks stay at 0.0% and then finally jump to 100.0% and finish. 
> Consequently, for jobs with small number of long-running map tasks, 
> the progress update is very coarse.
>
> As I understand, this progress meter (in absence of map tasks 
> explicitly setting the progress) was based on the RecordReader 
> reporting of how much of the current split has been read. Is this 
> something that got broken on the way? If not, what's the reason for 
> this, and how to fix it?
>

Does anyone have a suggestion about this problem? It's rather irritating 
- long-running tasks seem to be stuck at 0%, and only jump to 100% at 
the end of the task. This happens with 0.11.2 as well.

-- 
Best regards,
Andrzej Bialecki     <><
 ___. ___ ___ ___ _ _   __________________________________
[__ || __|__/|__||\/|  Information Retrieval, Semantic Web
___|||__||  \|  ||  |  Embedded Unix, System Integration
http://www.sigram.com  Contact: info at sigram dot com



Mime
View raw message