hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-3568) Optimize Job's progress calculations in MR AM
Date Thu, 15 Dec 2011 22:15:32 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-3568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Vinod Kumar Vavilapalli updated MAPREDUCE-3568:
-----------------------------------------------

          Component/s: mr-am
          Description: Besides catering to client requests, Job progress is calculated in
every heartbeat to the RM so as to print the MR AM's progress. Today the map and reduce progresses
are calculated by looking up of each task in a big map while we can simply make do with a
scan and aggregate. With large number of tasks, this can make a difference.
    Affects Version/s: 0.23.0
        Fix Version/s: 0.23.1
    
> Optimize Job's progress calculations in MR AM
> ---------------------------------------------
>
>                 Key: MAPREDUCE-3568
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3568
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: mr-am, mrv2, performance
>    Affects Versions: 0.23.0
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Vinod Kumar Vavilapalli
>             Fix For: 0.23.1
>
>
> Besides catering to client requests, Job progress is calculated in every heartbeat to
the RM so as to print the MR AM's progress. Today the map and reduce progresses are calculated
by looking up of each task in a big map while we can simply make do with a scan and aggregate.
With large number of tasks, this can make a difference.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message