hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Inigo Goiri (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5215) Scheduling containers based on external load in the servers
Date Thu, 09 Jun 2016 16:39:21 GMT

    [ https://issues.apache.org/jira/browse/YARN-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15322829#comment-15322829

Inigo Goiri commented on YARN-5215:

[~jlowe], my view on this feature is more like making the scheduler our of a "container" that
is not managed by YARN. This is a dynamic size container. Before moving forward with this,
we should agree that this is a proper abstraction and complementary to the overcommit effort.

Regarding the UI stuff, let's try to make a list of what to expose and I'll give it a try.
My first idea is to expose the following per node/queue/cluster:
* Node utilization
* Container utilization
* External utilization
* Allocated/unallocated resources

Can you think on any other metric/aggregation we should be reporting?

> Scheduling containers based on external load in the servers
> -----------------------------------------------------------
>                 Key: YARN-5215
>                 URL: https://issues.apache.org/jira/browse/YARN-5215
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Inigo Goiri
>         Attachments: YARN-5215.000.patch, YARN-5215.001.patch
> Currently YARN runs containers in the servers assuming that they own all the resources.
The proposal is to use the utilization information in the node and the containers to estimate
how much is consumed by external processes and schedule based on this estimation.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message