hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Junping Du (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-291) [Umbrella] Dynamic resource configuration
Date Thu, 14 Nov 2013 06:31:35 GMT

    [ https://issues.apache.org/jira/browse/YARN-291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13822212#comment-13822212
] 

Junping Du commented on YARN-291:
---------------------------------

Hi Cindy, Thanks for comments. You are right that #2 is needed in both cases (graceful NM
decommission and NM maintenance). Previously, YARN-999 is designed to release allocated containers
immediately so client may need to trigger it after tracking a timeout. Now, it can be tracked
in RM side and triggered in next NM-RM heartbeat after timeout which can be new scope of YARN-999.
I plan to work on it after finishing protocol (YARN-312), CLI (YARN-313) and gracefully decommission
(YARN-914). If you think #2 blocking your current NM maintenance work, please feel free to
take YARN-999 and move it forward. I will help in review, test, bug fix, etc.   

> [Umbrella] Dynamic resource configuration
> -----------------------------------------
>
>                 Key: YARN-291
>                 URL: https://issues.apache.org/jira/browse/YARN-291
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: nodemanager, scheduler
>            Reporter: Junping Du
>            Assignee: Junping Du
>              Labels: features
>         Attachments: Elastic Resources for YARN-v0.2.pdf, YARN-291-AddClientRMProtocolToSetNodeResource-03.patch,
YARN-291-CoreAndAdmin.patch, YARN-291-JMXInterfaceOnNM-02.patch, YARN-291-OnlyUpdateWhenResourceChange-01-fix.patch,
YARN-291-YARNClientCommandline-04.patch, YARN-291-all-v1.patch, YARN-291-core-HeartBeatAndScheduler-01.patch
>
>
> The current Hadoop YARN resource management logic assumes per node resource is static
during the lifetime of the NM process. Allowing run-time configuration on per node resource
will give us finer granularity of resource elasticity. This allows Hadoop workloads to coexist
with other workloads on the same hardware efficiently, whether or not the environment is virtualized.
More background and design details can be found in attached proposal.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message