Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 15 Jun 2016 04:02:09 +0000 (UTC)
From: "Carlo Curino (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12976940.1465406817000.458.1465963329603@Atlassian.JIRA>
In-Reply-To: <JIRA.12976940.1465406817000@Atlassian.JIRA>
References: <JIRA.12976940.1465406817000@Atlassian.JIRA> <JIRA.12976940.1465406817130@arcas>
Subject: [jira] [Commented] (YARN-5215) Scheduling containers based on
 external load in the servers
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 15 Jun 2016 04:02:11 -0000


    [ https://issues.apache.org/jira/browse/YARN-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15331102#comment-15331102 ] 

Carlo Curino commented on YARN-5215:
------------------------------------

Regarding preemptable vs OPPORTUNISTIC, we had this conversation with [~kkaranasos] and [~asuresh], where containers types could be {{NON-PREEMPTABLE}}, {{PREEMPTABLE}}, {{OPPORTUNISTIC}}, describing basically increasing level of how likely is a task to be interrupted. 
For {{NON-PREEMPTABLE}} the system does everything it can not to interrupt (bar physical machine failures), {{PREEMPTABLE}} are containers with a low, but non-null chance of being interrupted by the scheduler (e.g., allocations above a queue capacity, but on dedicated resources), and {{OPPORTUNISTIC}} are tasks that have a high(er) chance of being interrupted as they run on the left-over capacity from other containers, or on overcommitted resources or, or other risky forms of resource harvesting (as in this JIRA).   

BTW [~elgoiri] I would suggest to turn this in an umbrella JIRA, and separate the sub-step you list [in the comment above | https://issues.apache.org/jira/browse/YARN-5215?focusedCommentId=15330649&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15330649]. Some are likely less controversial and can be settled/committed earlier, while other parts are too "interesting" to go in easily :-)


> Scheduling containers based on external load in the servers
> -----------------------------------------------------------
>
>                 Key: YARN-5215
>                 URL: https://issues.apache.org/jira/browse/YARN-5215
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Inigo Goiri
>         Attachments: YARN-5215.000.patch, YARN-5215.001.patch
>
>
> Currently YARN runs containers in the servers assuming that they own all the resources. The proposal is to use the utilization information in the node and the containers to estimate how much is consumed by external processes and schedule based on this estimation.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org