hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Haibo Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1015) FS should watch node resource utilization and allocate opportunistic containers if appropriate
Date Wed, 08 Nov 2017 00:05:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16243117#comment-16243117

Haibo Chen commented on YARN-1015:

bq.  Could the allowed resource for overallocation change later while the function is still
assignContainer only happens in the scheduler, which serializes scheduling requests, so we
be OK most of the time. 
The handling of container decrease/demotion, however, bypasses scheduler and allocates
RMContainers directly followed by an immediate swap of existing containers. It will cause
accounting of opportunistic containers to go up briefly, and if the scheduler happens to be
opportunistic containers in that window, less opportunistic containers will be allocated.
But the accounting
will drop shortly after AM pulls the updated containers, and we'll be able to allocate opportunistic
containers later.

> FS should watch node resource utilization and allocate opportunistic containers if appropriate
> ----------------------------------------------------------------------------------------------
>                 Key: YARN-1015
>                 URL: https://issues.apache.org/jira/browse/YARN-1015
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Arun C Murthy
>            Assignee: Haibo Chen
>         Attachments: YARN-1015-YARN-1011.00.patch, YARN-1015-YARN-1011.01.patch, YARN-1015-YARN-1011.02.patch,
> FS should looks at resource utilization of nodes (provided by NM in heartbeat) and allocate
opportunistic containers if the resource utilization of the node is below its allocation threshold.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message