hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-3901) lazy load JobHistory Task and TaskAttempt details
Date Tue, 28 Feb 2012 00:35:56 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-3901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Vinod Kumar Vavilapalli updated MAPREDUCE-3901:

       Resolution: Fixed
    Fix Version/s: 0.23.2
     Release Note: Modified JobHistory records in YARN to lazily load job and task reports
so as to improve UI response times.
     Hadoop Flags: Reviewed
           Status: Resolved  (was: Patch Available)

I just committed this to trunk, branch-0.23 and branch-0.23.2. Thanks Sid!
> lazy load JobHistory Task and TaskAttempt details
> -------------------------------------------------
>                 Key: MAPREDUCE-3901
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3901
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: jobhistoryserver, mrv2
>    Affects Versions: 0.23.0
>            Reporter: Siddharth Seth
>            Assignee: Siddharth Seth
>             Fix For: 0.23.2
>         Attachments: MR3901.txt, MR3901_v2.txt, MR3901_v3.txt
> The job history UI and MRClientProtocol calls routed via JobHistory are very slow for
large jobs. Some of this time is spent parsing the history file. A good chunk is spent pre-creating
lots of objects which may never be used. Those can be create when required - bringing down
the load times of job history pages and getJobReport etc calls to approximately the history
file parse time.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message