Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-issues@hadoop.apache.org
Date: Tue, 19 May 2015 01:03:02 +0000 (UTC)
From: "ericson yang (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Message-ID: <JIRA.12445613.1263547555000.151167.1431997382412@Atlassian.JIRA>
In-Reply-To: <JIRA.12445613.1263547555000@Atlassian.JIRA>
References: <JIRA.12445613.1263547555000@Atlassian.JIRA>
 <JIRA.12445613.1263547555823@arcas>
Subject: [jira] [Updated] (MAPREDUCE-1380) Adaptive Scheduler
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


     [ https://issues.apache.org/jira/browse/MAPREDUCE-1380?page=3Dcom.atla=
ssian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ericson yang updated MAPREDUCE-1380:
------------------------------------
    Assignee: Jord=C3=A0 Polo  (was: ericson yang)

> Adaptive Scheduler
> ------------------
>
>                 Key: MAPREDUCE-1380
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1380
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>    Affects Versions: 2.4.1
>            Reporter: Jord=C3=A0 Polo
>            Assignee: Jord=C3=A0 Polo
>            Priority: Minor
>         Attachments: MAPREDUCE-1380-branch-1.2.patch, MAPREDUCE-1380_0.1.=
patch, MAPREDUCE-1380_1.1.patch, MAPREDUCE-1380_1.1.pdf
>
>
> The Adaptive Scheduler is a pluggable Hadoop scheduler that automatically=
 adjusts the amount of used resources depending on the performance of jobs =
and on user-defined high-level business goals.
> Existing Hadoop schedulers are focused on managing large, static clusters=
 in which nodes are added or removed manually. On the other hand, the goal =
of this scheduler is to improve the integration of Hadoop and the applicati=
ons that run on top of it with environments that allow a more dynamic provi=
sioning of resources.
> The current implementation is quite straightforward. Users specify a dead=
line at job submission time, and the scheduler adjusts the resources to mee=
t that deadline (at the moment, the scheduler can be configured to either m=
inimize or maximize the amount of resources). If multiple jobs are run simu=
ltaneously, the scheduler prioritizes them by deadline. Note that the curre=
nt approach to estimate the completion time of jobs is quite simplistic: it=
 is based on the time it takes to finish each task, so it works well with r=
egular jobs, but there is still room for improvement for unpredictable jobs=
.
> The idea is to further integrate it with cloud-like and virtual environme=
nts (such as Amazon EC2, Emotive, etc.) so that if, for instance, a job isn=
't able to meet its deadline, the scheduler automatically requests more res=
ources.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)