hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ray Chiang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-6394) Speed up Task processing loop in HsTasksBlock#render()
Date Fri, 10 Jul 2015 22:41:05 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-6394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ray Chiang updated MAPREDUCE-6394:
----------------------------------
    Attachment: MAPREDUCE-6394.004.patch

- Change direct setCounters()/getCounters() to use late-binding combo setRawCounters()/getCounters()
for conversion.
- Profiling-wise for 751k tasks, the loop in HsTasksBlock#render() takes ~100-110 seconds
without this fix and around 20-25 seconds with this fix.

> Speed up Task processing loop in HsTasksBlock#render()
> ------------------------------------------------------
>
>                 Key: MAPREDUCE-6394
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6394
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: jobhistoryserver
>    Affects Versions: 2.7.0
>            Reporter: Ray Chiang
>            Assignee: Ray Chiang
>              Labels: supportability
>         Attachments: MAPREDUCE-6394.001.patch, MAPREDUCE-6394.002.patch, MAPREDUCE-6394.003.patch,
MAPREDUCE-6394.004.patch
>
>
> In HsTasksBlock#render(), there is a loop to create a Javascript table which slows down
immensely on jobs with a large number of tasks (200k or more).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message