hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Scott Chen (JIRA)" <j...@apache.org>
Subject [jira] Updated: (MAPREDUCE-1198) Alternatively schedule different types of tasks in fair share scheduler
Date Mon, 16 Nov 2009 20:12:39 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Scott Chen updated MAPREDUCE-1198:

    Attachment: MAPREDUCE-1198-v1.patch

In FairScheduler.assignTasks(), before scanning the jobs for a map and a reduce, we first
sort the task type order according to the number of running maps and reduces on the TT. The
original way will always first look at a map and then a reduce which may starve reduce tasks.

This is basically an implementation of Matei's pseudo code mentioned in MAPREDUCE-961.

> Alternatively schedule different types of tasks in fair share scheduler
> -----------------------------------------------------------------------
>                 Key: MAPREDUCE-1198
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1198
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: contrib/fair-share
>            Reporter: Scott Chen
>            Assignee: Scott Chen
>         Attachments: MAPREDUCE-1198-v1.patch
> Matei has mentioned in MAPREDUCE-961 that the current scheduler will first try to launch
map tasks until canLaunthTask() returns false then look for reduce tasks. This might starve
reduce task. He also mention that alternatively schedule different types of tasks can solve
this problem.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message