hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cindy Li (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-291) [Umbrella] Dynamic resource configuration
Date Mon, 02 Dec 2013 20:38:38 GMT

    [ https://issues.apache.org/jira/browse/YARN-291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13836883#comment-13836883

Cindy Li commented on YARN-291:

Junping, just saw your comments on YARN-999 .  I can help on it. 
Can you help me understand the use cases/scope of YARN-999 besides graceful decommission.
In the code below:

      // TODO process resource over-commitment case (allocated containers
      // > total capacity) in different option by getting value of
      // overCommitTimeoutMillis.

By different options above, do you mean overCommitTimeoutMills > 0, = 0, <0 ? I want
to find out more use cases associated with this setting besides graceful decommission. For
example, you mentioned preemption for long running tasks in YARN-999, is that part of or a
different use case from graceful decommission?

Also, about the August patch CoreAndAdmin.patch (in YARN-291) , can you let us know your plan
about it because it seems useful for graceful decommission from outside of YARN code. 


> [Umbrella] Dynamic resource configuration
> -----------------------------------------
>                 Key: YARN-291
>                 URL: https://issues.apache.org/jira/browse/YARN-291
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: nodemanager, scheduler
>            Reporter: Junping Du
>            Assignee: Junping Du
>              Labels: features
>         Attachments: Elastic Resources for YARN-v0.2.pdf, YARN-291-AddClientRMProtocolToSetNodeResource-03.patch,
YARN-291-CoreAndAdmin.patch, YARN-291-JMXInterfaceOnNM-02.patch, YARN-291-OnlyUpdateWhenResourceChange-01-fix.patch,
YARN-291-YARNClientCommandline-04.patch, YARN-291-all-v1.patch, YARN-291-core-HeartBeatAndScheduler-01.patch
> The current Hadoop YARN resource management logic assumes per node resource is static
during the lifetime of the NM process. Allowing run-time configuration on per node resource
will give us finer granularity of resource elasticity. This allows Hadoop workloads to coexist
with other workloads on the same hardware efficiently, whether or not the environment is virtualized.
More background and design details can be found in attached proposal.

This message was sent by Atlassian JIRA

View raw message