hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun Suresh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3656) LowCost: A Cost-Based Placement Agent for YARN Reservations
Date Sat, 11 Jul 2015 07:56:05 GMT

    [ https://issues.apache.org/jira/browse/YARN-3656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14623300#comment-14623300

Arun Suresh commented on YARN-3656:

[~jyaniv], [~curino] and [~imenache], This looks really interesting optimization to the existing
reservation algorithm. Thanks for working on this !!

I took an initial pass at the latest patch. Couple of minor nits :
* TryManyReservationAgents.java
in both create an updateReservation:
line 85/54 : you don’t need the if.. just return alg.update/create..
* minor suggestion : was wondering if instead of a TryManyReservationAgents, allow ReservationAgent
itself to have a fallbackReservationAgent. An agent can then call its fallback (and if that
fails, the fallback’s fallback etc.. until no more fallbacks exist). That way, you don't
really need to maintain a LinkedList etc.

Will spend some time on the paper and review the actual Algorithm over the weekend..

> LowCost: A Cost-Based Placement Agent for YARN Reservations
> -----------------------------------------------------------
>                 Key: YARN-3656
>                 URL: https://issues.apache.org/jira/browse/YARN-3656
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: capacityscheduler, resourcemanager
>    Affects Versions: 2.6.0
>            Reporter: Ishai Menache
>            Assignee: Jonathan Yaniv
>              Labels: capacity-scheduler, resourcemanager
>         Attachments: LowCostRayonExternal.pdf, YARN-3656-v1.1.patch, YARN-3656-v1.2.patch,
YARN-3656-v1.patch, lowcostrayonexternal_v2.pdf
> YARN-1051 enables SLA support by allowing users to reserve cluster capacity ahead of
time. YARN-1710 introduced a greedy agent for placing user reservations. The greedy agent
makes fast placement decisions but at the cost of ignoring the cluster committed resources,
which might result in blocking the cluster resources for certain periods of time, and in turn
rejecting some arriving jobs.
> We propose LowCost – a new cost-based planning algorithm. LowCost “spreads” the
demand of the job throughout the allowed time-window according to a global, load-based cost

This message was sent by Atlassian JIRA

View raw message