hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-3568) Optimize Job's progress calculations in MR AM
Date Tue, 27 Dec 2011 19:56:30 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-3568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Vinod Kumar Vavilapalli updated MAPREDUCE-3568:

      Resolution: Fixed
    Release Note: Optimized Job's progress calculations in MR AM.
          Status: Resolved  (was: Patch Available)

Just committed this to trunk and branch-0.23.
> Optimize Job's progress calculations in MR AM
> ---------------------------------------------
>                 Key: MAPREDUCE-3568
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3568
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: mr-am, mrv2, performance
>    Affects Versions: 0.23.0
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Vinod Kumar Vavilapalli
>            Priority: Critical
>             Fix For: 0.23.1
>         Attachments: MAPREDUCE-3568-20111215.1.txt, MAPREDUCE-3568-20111220.txt, MAPREDUCE-3568-20111222.txt,
> Besides catering to client requests, Job progress is calculated in every heartbeat to
the RM so as to print the MR AM's progress. Today the map and reduce progresses are calculated
by looking up of each task in a big map while we can simply make do with a scan and aggregate.
With large number of tasks, this can make a difference.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message