hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anubhav Dhoot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3920) FairScheduler Reserving a node for a container should be configurable to allow it used only for large containers
Date Wed, 05 Aug 2015 22:02:05 GMT

    [ https://issues.apache.org/jira/browse/YARN-3920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14659048#comment-14659048
] 

Anubhav Dhoot commented on YARN-3920:
-------------------------------------

Yeah that should be possible to do when thats the intention and thats ok. I am trying to avoid
cases when its not the intention to disable reservation but it happens due to environment+configuration.

In ratio of max case, one can disable reservation only by setting the ratio to above 1.0.

In the multiple of increment case, one can disable it by setting say setting a multiple to
 say 4 times increment and your max on the cluster (which depends on the min node size of
cluster in addition to max config) is 3 times the increment. That can happen even if it was
not your intention to disable it.


> FairScheduler Reserving a node for a container should be configurable to allow it used
only for large containers
> ----------------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-3920
>                 URL: https://issues.apache.org/jira/browse/YARN-3920
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: fairscheduler
>            Reporter: Anubhav Dhoot
>            Assignee: Anubhav Dhoot
>         Attachments: yARN-3920.001.patch, yARN-3920.002.patch
>
>
> Reserving a node for a container was designed for preventing large containers from starvation
from small requests that keep getting into a node. Today we let this be used even for a small
container request. This has a huge impact on scheduling since we block other scheduling requests
until that reservation is fulfilled. We should make this configurable so its impact can be
minimized by limiting it for large container requests as originally intended. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message