hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3901) lazy load JobHistory Task and TaskAttempt details
Date Sun, 04 Mar 2012 00:29:58 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13221754#comment-13221754

Vinod Kumar Vavilapalli commented on MAPREDUCE-3901:

bq. After this, I think we can have separate caches for list of CompletedJob with tasks loaded
and list of CompletedJob with no tasks loaded. Will open a separate ticket.
Filed MAPREDUCE-3966.
> lazy load JobHistory Task and TaskAttempt details
> -------------------------------------------------
>                 Key: MAPREDUCE-3901
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3901
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: jobhistoryserver, mrv2
>    Affects Versions: 0.23.0
>            Reporter: Siddharth Seth
>            Assignee: Siddharth Seth
>             Fix For: 0.23.2
>         Attachments: MR3901.txt, MR3901_v2.txt, MR3901_v3.txt
> The job history UI and MRClientProtocol calls routed via JobHistory are very slow for
large jobs. Some of this time is spent parsing the history file. A good chunk is spent pre-creating
lots of objects which may never be used. Those can be create when required - bringing down
the load times of job history pages and getJobReport etc calls to approximately the history
file parse time.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message