hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Subru Krishnan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5261) Lease/Reclaim Extension to Yarn
Date Sat, 16 Jul 2016 02:16:20 GMT

    [ https://issues.apache.org/jira/browse/YARN-5261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15380427#comment-15380427

Subru Krishnan commented on YARN-5261:

@yuuu' thanks for the clarifications.

I understand there are slight nuances for your use case but generally it's good to consolidate
related efforts as otherwise we will have an explosion of specialized features in YARN (already
see a trend). IIUC, this is what [~jlowe] and [~sunilg] are both alluding to.

We do have the ability to reserve cluster resources as part of YARN-1051/YARN-2572/YARN-2573.
Would that be sufficient or do you need to be able to do it at node level? If the latter is
the case, you can consider achieving it in combination of reservation system with node labels
(YARN-4193), intuition is each node that can be leased has a label (which can be it's name).

Another option is to have LeaseManager run outside of YARN initially and use the dynamic node
resource configuration (YARN-291) to affect the change you want.


> Lease/Reclaim Extension to Yarn
> -------------------------------
>                 Key: YARN-5261
>                 URL: https://issues.apache.org/jira/browse/YARN-5261
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: scheduler
>            Reporter: Yu
>         Attachments: YARN-5261-1.patch, YARN-5261-2.patch, YARN-5261-3.patch, Yarn-5261.pdf
> In some clusters outside of Yarn, the machines resources are not fully utilized, e.g.,
resource usage is quite low at night.  
> To better utilize the resources while keep the existing SLA of the cluster, Lease/Reclaim
Extension to Yarn is introduced.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message