hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2440) Cgroups should allow YARN containers to be limited to allocated cores
Date Tue, 09 Sep 2014 15:16:30 GMT

    [ https://issues.apache.org/jira/browse/YARN-2440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14127086#comment-14127086

Jason Lowe commented on YARN-2440:

bq. Specifically in the context of heterogeneous clusters where uniform % configurations can
go really bad where the only resort will then be to do per-node configuration - not ideal.

Yes, I could see the heterogenous cluster being a case where specifying absolute instead of
relative may be desirable.  My biggest concern is that it's confusing when trying to combine
the absolute and relative concepts -- it's not obvious if one overrides the other or if one
is relative to the other.

Part of my concern to keep this as simple as possible and the configuration burden to an absolute
minimum is that I'm missing the real-world use case.  As I mentioned before, I think most
users would rather not use the functionality proposed by this JIRA but instead setup peer
cgroups for other systems and set their relative cgroup shares appropriately.  With this JIRA
the CPUs could sit idle despite demand from YARN containers, while a peer cgroup setup allows
CPU guarantees without idle CPUs if the demand is there.

> Cgroups should allow YARN containers to be limited to allocated cores
> ---------------------------------------------------------------------
>                 Key: YARN-2440
>                 URL: https://issues.apache.org/jira/browse/YARN-2440
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Varun Vasudev
>            Assignee: Varun Vasudev
>         Attachments: apache-yarn-2440.0.patch, apache-yarn-2440.1.patch, apache-yarn-2440.2.patch,
apache-yarn-2440.3.patch, apache-yarn-2440.4.patch, screenshot-current-implementation.jpg
> The current cgroups implementation does not limit YARN containers to the cores allocated
in yarn-site.xml.

This message was sent by Atlassian JIRA

View raw message