hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tamas Sarlos (JIRA)" <j...@apache.org>
Subject [jira] Updated: (MAPREDUCE-1306) [MUMAK] Randomize the arrival of heartbeat responses
Date Tue, 09 Mar 2010 08:11:27 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-1306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Tamas Sarlos updated MAPREDUCE-1306:
------------------------------------

    Attachment: MAPREDUCE-1306-20100308.patch

Attaching the updated patch that implements the random seeding option for mumak and rumen.


In order to make the simulation deterministic HashSets and HashMaps need to be replaced with
different collection classes since they make no guarantees as to the order of iteration. E.g.
JobInProgress iterates over the JobTrackers's nodesAtMaxLevel HashSet, the order of this iteration
influences the scheduling of non-local maps. Using AspectJ all HashSets and HashMaps are replaced
with LinkedHashSets and LinkedHashMaps, whose iteration order is determined by the order of
insertions. This solution needs to be revisited if mumak ever becomes multi-threaded. An added
functional test verifies deterministic replay by comparing the job history files.

> [MUMAK] Randomize the arrival of heartbeat responses
> ----------------------------------------------------
>
>                 Key: MAPREDUCE-1306
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1306
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: contrib/mumak
>    Affects Versions: 0.21.0, 0.22.0
>            Reporter: Tamas Sarlos
>            Assignee: Tamas Sarlos
>             Fix For: 0.21.0, 0.22.0
>
>         Attachments: MAPREDUCE-1306-20100108.patch, MAPREDUCE-1306-20100308.patch
>
>
> We propose to make the following changes to mumak, MAPREDUCE-728
> - make the timing of heartbeat responses more realistic by adding an option to randomly
perturb them
> - randomize the startup time of task trackers in a fixed interval 
> - remove 2 magic constants from SimulatorEngine and make sure that the first job is submitted
only after the entire cluster is up and running

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message