hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun Suresh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1015) FS should watch node resource utilization and allocate opportunistic containers if appropriate
Date Tue, 07 Nov 2017 00:09:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16241212#comment-16241212

Arun Suresh commented on YARN-1015:

Thanks for the patch [~haibochen]

So couple of comments.
* I assume the scheduler will attempt to allocate OPP containers only after all reserved containers
AND after it has allocated GUAR containers on the node - In which case, I don't think you
should even attempt to promote the container in call (I was referring to the TODO you added).
* I think a better way to auto-promote would be at the time a guaranteed container completes,
the RM can check if there an OPP containers of the app which can be promoted - need not be
the same app, it can also possibly be another app from the same queue - or we can go the the
other way, and decide to promote locally at the NM if we know that an app has running opportunistic
container on the same node where it's guaranteed container has completed.

> FS should watch node resource utilization and allocate opportunistic containers if appropriate
> ----------------------------------------------------------------------------------------------
>                 Key: YARN-1015
>                 URL: https://issues.apache.org/jira/browse/YARN-1015
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Arun C Murthy
>            Assignee: Haibo Chen
>         Attachments: YARN-1015-YARN-1011.00.patch, YARN-1015-YARN-1011.01.patch, YARN-1015-YARN-1011.prelim.patch
> FS should looks at resource utilization of nodes (provided by NM in heartbeat) and allocate
opportunistic containers if the resource utilization of the node is below its allocation threshold.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message