Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 9 Nov 2016 22:31:59 +0000 (UTC)
From: "Miklos Szegedi (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.13019005.1478554169000.237223.1478730719043@Atlassian.JIRA>
In-Reply-To: <JIRA.13019005.1478554169000@Atlassian.JIRA>
References: <JIRA.13019005.1478554169000@Atlassian.JIRA> <JIRA.13019005.1478554169045@arcas>
Subject: [jira] [Updated] (YARN-5849) Automatically create YARN control
 group for pre-mounted cgroups
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 09 Nov 2016 22:32:00 -0000


     [ https://issues.apache.org/jira/browse/YARN-5849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Miklos Szegedi updated YARN-5849:
---------------------------------
    Attachment: YARN-5849.002.patch

I updated the patch with a better implementation based on your comments. CGroupsHandlerImpl used to have a single mountCGroupController function to handle the case when mount is enabled. With this change there will be a function called initializeCGroupController. It will call initializePreMountedCGroupController to make sure the group exist in the pre-mounted directories. Otherwise, it calls mountCGroupController, if we are dealing with a pre-mounted controller. All resource handlers are supposed to call initializeCGroupController as they do with mountCGroupController now. This way initializeCGroupController is called once only if the subsystem in question is enabled ensuring that we do not touch the cgroup mount points of external subsystems.

> Automatically create YARN control group for pre-mounted cgroups
> ---------------------------------------------------------------
>
>                 Key: YARN-5849
>                 URL: https://issues.apache.org/jira/browse/YARN-5849
>             Project: Hadoop YARN
>          Issue Type: Improvement
>    Affects Versions: 2.7.3, 3.0.0-alpha1, 3.0.0-alpha2
>            Reporter: Miklos Szegedi
>            Assignee: Miklos Szegedi
>            Priority: Minor
>         Attachments: YARN-5849.000.patch, YARN-5849.001.patch, YARN-5849.002.patch
>
>
> Yarn can be launched with linux-container-executor.cgroups.mount set to false. It will search for the cgroup mount paths set up by the administrator parsing the /etc/mtab file. You can also specify resource.percentage-physical-cpu-limit to limit the CPU resources assigned to containers.
> linux-container-executor.cgroups.hierarchy is the root of the settings of all YARN containers. If this is specified but not created YARN will fail at startup:
> Caused by: java.io.FileNotFoundException: /cgroups/cpu/hadoop-yarn/cpu.cfs_period_us (Permission denied)
> org.apache.hadoop.yarn.server.nodemanager.util.CgroupsLCEResourcesHandler.updateCgroup(CgroupsLCEResourcesHandler.java:263)
> This JIRA is about automatically creating YARN control group in the case above. It reduces the cost of administration.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org