hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5900) Configuring minimum-allocation-mb at queue level
Date Thu, 17 Nov 2016 14:17:58 GMT

    [ https://issues.apache.org/jira/browse/YARN-5900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15673838#comment-15673838

Jason Lowe commented on YARN-5900:

I don't believe this would simplify anything, rather just make it more complicated within
the scheduler and more complicated to explain to users what's going on.  In addition it will
add to internal fragmentation of container allocations, since we will force container allocations
that are smaller than the normal minimum to a larger value even though the task within the
container will not use it, reducing the overall utilization of the cluster.

> Configuring minimum-allocation-mb at queue level
> ------------------------------------------------
>                 Key: YARN-5900
>                 URL: https://issues.apache.org/jira/browse/YARN-5900
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Manikandan R
> Motivation for proposing minimum-allocation-mb at queue level in the form of 
> yarn.scheduler.capacity.<queue-path>.minimum-allocation-mb
> is, when the queue structure has been designed on resource usages (only memory for now).
For example, there could be three segments like small, medium & large jobs and queues
can be created for each segment accordingly. With this, it would be good to configure the
min container size of each queue separately. For example, For small, it is 1 GB, For medium,
it is 3 GB, For large, it can be 6 GB. 
> Would this simplify container release process and its overall management, eventually
reduces no. of containers running at any moment?

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message