hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jim Brennan (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-8656) container-executor should not write cgroup tasks files for docker containers
Date Tue, 14 Aug 2018 13:23:00 GMT

     [ https://issues.apache.org/jira/browse/YARN-8656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Jim Brennan updated YARN-8656:
    Attachment: YARN-8656.001.patch

> container-executor should not write cgroup tasks files for docker containers
> ----------------------------------------------------------------------------
>                 Key: YARN-8656
>                 URL: https://issues.apache.org/jira/browse/YARN-8656
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Jim Brennan
>            Assignee: Jim Brennan
>            Priority: Major
>              Labels: Docker
>         Attachments: YARN-8656.001.patch
> If cgroups are enabled, we pass the {{--cgroup-parent}} option to {{docker run}} to ensure
that all processes for the container are placed into a cgroup under (for example) {{/sys/fs/cgroup/cpu/cgroups.hierarchy/container_id}}.
Docker creates a cgroup there with the docker container id as the name and all of the processes
in the container go into that cgroup.
> container-executor has code in {{launch_docker_container_as_user()}} that then cherry-picks
the PID of the docker container (usually the launch shell) and writes that into the {{/sys/fs/cgroup/cpu/cgroups.hierarchy/container_id/tasks}} file,
effectively moving it from {{/sys/fs/cgroup/cpu/cgroups.hierarchy/container_id/docker_container_id}} to {{/sys/fs/cgroup/cpu/cgroups.hierarchy/container_id}}. 
So you end up with one process out of the container in the {{container_id}} cgroup, and
the rest in the {{container_id/docker_container_id}} cgroup.
> Since we are passing the {{--cgroup-parent}} to docker, there is no need to manually
write the container pid to the tasks file - we can just remove the code that does this in
the docker case.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message