hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Billy Pearson (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4000) Reduce progress in the Reducer crosses 1
Date Thu, 12 Feb 2009 01:07:59 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12672848#action_12672848
] 

Billy Pearson commented on HADOOP-4000:
---------------------------------------

I see this also when using streaming. the reduce map disappears because over 100% task

> Reduce progress in the Reducer crosses 1
> ----------------------------------------
>
>                 Key: HADOOP-4000
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4000
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Amar Kamat
>
> I printed the progress in the {{Progress.java}} for every progress made and I found this
scenario
> {noformat}
> update called for reduce with progress 0.99998355
> update called for reduce with progress 0.99998355
> update called for reduce with progress 1.0000043
> update called for reduce with progress 1.0000043
> update called for reduce with progress 1.0000244
> {noformat}
> Following are the lines I added to get this 
> {code:title=Progress.java|borderStyle=solid}
> /** Called during execution on a leaf node to set its progress. */
>   public synchronized void set(float progress) {
>     System.out.println("update called for " + status + " with progress " + progress);
>     this.progress = progress;
>   }
> {code}
> This needs investigation and will also explain why occasionally users see jobs that are
complete and their progress less that 100%.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message