hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Inigo Goiri (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5215) Scheduling containers based on external load in the servers
Date Wed, 15 Jun 2016 20:28:09 GMT

    [ https://issues.apache.org/jira/browse/YARN-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332483#comment-15332483

Inigo Goiri commented on YARN-5215:

[~kkaranasos] I kind of like the "fake container" approach. In addition, that would expose
the real status of the cluster to the users and I think it would cover [~jlowe]'s concern.
For distinguishing fake containers, we could make them be part of a fake application and replace
the framework in the Application Type to something like EXTERNAL and the actual external load
as the application name. We should leverage the unmanaged AM concept.

I also like this approach because we can create a default service that does the same thing
I'm proposing in this patch and just update the size of the container based on node and containers

> Scheduling containers based on external load in the servers
> -----------------------------------------------------------
>                 Key: YARN-5215
>                 URL: https://issues.apache.org/jira/browse/YARN-5215
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Inigo Goiri
>         Attachments: YARN-5215.000.patch, YARN-5215.001.patch
> Currently YARN runs containers in the servers assuming that they own all the resources.
The proposal is to use the utilization information in the node and the containers to estimate
how much is consumed by external processes and schedule based on this estimation.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message