hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Haibo Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1015) FS should watch node resource utilization and allocate opportunistic containers if appropriate
Date Wed, 08 Nov 2017 00:05:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16243117#comment-16243117
] 

Haibo Chen commented on YARN-1015:
----------------------------------

bq.  Could the allowed resource for overallocation change later while the function is still
running?
assignContainer only happens in the scheduler, which serializes scheduling requests, so we
should
be OK most of the time. 
The handling of container decrease/demotion, however, bypasses scheduler and allocates
RMContainers directly followed by an immediate swap of existing containers. It will cause
the 
accounting of opportunistic containers to go up briefly, and if the scheduler happens to be
allocating
opportunistic containers in that window, less opportunistic containers will be allocated.
But the accounting
will drop shortly after AM pulls the updated containers, and we'll be able to allocate opportunistic
containers later.

> FS should watch node resource utilization and allocate opportunistic containers if appropriate
> ----------------------------------------------------------------------------------------------
>
>                 Key: YARN-1015
>                 URL: https://issues.apache.org/jira/browse/YARN-1015
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Arun C Murthy
>            Assignee: Haibo Chen
>         Attachments: YARN-1015-YARN-1011.00.patch, YARN-1015-YARN-1011.01.patch, YARN-1015-YARN-1011.02.patch,
YARN-1015-YARN-1011.prelim.patch
>
>
> FS should looks at resource utilization of nodes (provided by NM in heartbeat) and allocate
opportunistic containers if the resource utilization of the node is below its allocation threshold.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message