hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amar Kamat (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-2591) [Gridmix] Improve cumulative CPU usage emulation
Date Tue, 14 Jun 2011 13:58:47 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049186#comment-13049186

Amar Kamat commented on MAPREDUCE-2591:

Adding details:
1. JVM reuse: Here the total cpu usage of the process tree will be a net sum of all the other
tasks that ran in the current JVM. This information is not known to the task. Cpu usage emulator
should identify such cases and emulate on top of the current cpu usage. 
2. Short lived maps emulating high CPU usage: If the total span of the map task is short,
then the task will exit before emulating the resource usage. The framework (or the emulator)
should detect such situations (based on map input data size, input records, features like
compression etc) and emulate aggressively. On way to do this would be to give more weight
in the early phases of the task.
3. Long running maps/reducers with high CPU usage: If the total runtime of the task is high
and the total expected cpu usage is high, then the emulator should speed up its emulation
phase. One option can be to use multi-threading. Other option would be to switch to a more
cpu intensive operation(s).

> [Gridmix] Improve cumulative CPU usage emulation
> ------------------------------------------------
>                 Key: MAPREDUCE-2591
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2591
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: contrib/gridmix
>            Reporter: Amar Kamat
>            Assignee: Amar Kamat
>             Fix For: 0.23.0
> MAPREDUCE-2106 introduced a basic cumulative cpu usage emulation in Gridmix. There are
some known cases where it can be improved. The cases are
> 1. JVM reuse
> 2. Short lived maps emulating high CPU usage
> 3. Long running maps/reducers with high CPU usage

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message