hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (Work stopped) (JIRA)" <j...@apache.org>
Subject [jira] [Work stopped] (MAPREDUCE-3568) Optimize Job's progress calculations in MR AM
Date Wed, 21 Dec 2011 00:57:31 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-3568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Work on MAPREDUCE-3568 stopped by Vinod Kumar Vavilapalli.

> Optimize Job's progress calculations in MR AM
> ---------------------------------------------
>
>                 Key: MAPREDUCE-3568
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3568
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: mr-am, mrv2, performance
>    Affects Versions: 0.23.0
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Vinod Kumar Vavilapalli
>             Fix For: 0.23.1
>
>         Attachments: MAPREDUCE-3568-20111215.1.txt, MAPREDUCE-3568-20111220.txt
>
>
> Besides catering to client requests, Job progress is calculated in every heartbeat to
the RM so as to print the MR AM's progress. Today the map and reduce progresses are calculated
by looking up of each task in a big map while we can simply make do with a scan and aggregate.
With large number of tasks, this can make a difference.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message