hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Haibo Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1015) FS should watch node resource utilization and allocate opportunistic containers if appropriate
Date Tue, 07 Nov 2017 01:14:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16241281#comment-16241281
] 

Haibo Chen commented on YARN-1015:
----------------------------------

Thanks [~asuresh] for the review!
bq.  I don't think you should even attempt to promote the container in call. 
Right. Will remove the TODO comment to avoid confusion. 

Given our offline discussion, we could unify scheduling of reserved, guaranteed and opportunistic
containers.
That is, we can rely on the ordering of scheduling requests of reserved guaranteed, opportunistic
containers,
like we do in container increase/promotion.  Each time we allocate an opportunistic container,
we'd also add
a scheduler  request for the promotion immediately. The new request will be ordered with existing
requests
in a FIFO fashion. At the next scheduling opportunity, after we finish allocating guaranteed
containers, we'd
check if any of the satisfied scheduling requests is for container promotion, and promote
the container explicitly.
Thoughts?

The reason why we want to order requests in a FIFO fashion is that we want to guarantee to
users that if you
are willing to accept an OPP container for your guaranteed requests, you can start running
things early. For 
example, if you just want GUARANTEED container, you'll need to wait for  2 mins, but if you
are OK with
oversubscription, you can start right now as an OPP container and will be promoted 2 mins
later. 

> FS should watch node resource utilization and allocate opportunistic containers if appropriate
> ----------------------------------------------------------------------------------------------
>
>                 Key: YARN-1015
>                 URL: https://issues.apache.org/jira/browse/YARN-1015
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Arun C Murthy
>            Assignee: Haibo Chen
>         Attachments: YARN-1015-YARN-1011.00.patch, YARN-1015-YARN-1011.01.patch, YARN-1015-YARN-1011.prelim.patch
>
>
> FS should looks at resource utilization of nodes (provided by NM in heartbeat) and allocate
opportunistic containers if the resource utilization of the node is below its allocation threshold.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message