hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Miklos Szegedi (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-5849) Automatically create YARN control group for pre-mounted cgroups
Date Wed, 09 Nov 2016 22:31:59 GMT

     [ https://issues.apache.org/jira/browse/YARN-5849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Miklos Szegedi updated YARN-5849:
---------------------------------
    Attachment: YARN-5849.002.patch

I updated the patch with a better implementation based on your comments. CGroupsHandlerImpl
used to have a single mountCGroupController function to handle the case when mount is enabled.
With this change there will be a function called initializeCGroupController. It will call
initializePreMountedCGroupController to make sure the group exist in the pre-mounted directories.
Otherwise, it calls mountCGroupController, if we are dealing with a pre-mounted controller.
All resource handlers are supposed to call initializeCGroupController as they do with mountCGroupController
now. This way initializeCGroupController is called once only if the subsystem in question
is enabled ensuring that we do not touch the cgroup mount points of external subsystems.

> Automatically create YARN control group for pre-mounted cgroups
> ---------------------------------------------------------------
>
>                 Key: YARN-5849
>                 URL: https://issues.apache.org/jira/browse/YARN-5849
>             Project: Hadoop YARN
>          Issue Type: Improvement
>    Affects Versions: 2.7.3, 3.0.0-alpha1, 3.0.0-alpha2
>            Reporter: Miklos Szegedi
>            Assignee: Miklos Szegedi
>            Priority: Minor
>         Attachments: YARN-5849.000.patch, YARN-5849.001.patch, YARN-5849.002.patch
>
>
> Yarn can be launched with linux-container-executor.cgroups.mount set to false. It will
search for the cgroup mount paths set up by the administrator parsing the /etc/mtab file.
You can also specify resource.percentage-physical-cpu-limit to limit the CPU resources assigned
to containers.
> linux-container-executor.cgroups.hierarchy is the root of the settings of all YARN containers.
If this is specified but not created YARN will fail at startup:
> Caused by: java.io.FileNotFoundException: /cgroups/cpu/hadoop-yarn/cpu.cfs_period_us
(Permission denied)
> org.apache.hadoop.yarn.server.nodemanager.util.CgroupsLCEResourcesHandler.updateCgroup(CgroupsLCEResourcesHandler.java:263)
> This JIRA is about automatically creating YARN control group in the case above. It reduces
the cost of administration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message