hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-8784) DockerLinuxContainerRuntime prevents access to distributed cache entries on a full disk
Date Wed, 19 Sep 2018 22:26:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-8784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16621276#comment-16621276

Hudson commented on YARN-8784:

SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #15025 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/15025/])
YARN-8784. DockerLinuxContainerRuntime prevents access to distributed (jlowe: rev 6b5838ed3220f992092c7348f92f1d9d0d4a3061)
* (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java
* (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java

> DockerLinuxContainerRuntime prevents access to distributed cache entries on a full disk
> ---------------------------------------------------------------------------------------
>                 Key: YARN-8784
>                 URL: https://issues.apache.org/jira/browse/YARN-8784
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>    Affects Versions: 3.2.0, 3.1.1
>            Reporter: Jason Lowe
>            Assignee: Eric Badger
>            Priority: Major
>              Labels: Docker
>             Fix For: 3.2.0, 3.1.2
>         Attachments: YARN-8784.001.patch
> DockerLinuxContainerRuntime bind mounts the filecache and usercache directories into
the container to allow tasks to access entries in the distributed cache.  However it only
bind mounts  directories on disks that are considered good, and disks that are full or bad
are not in that list.  If a container tries to run with a distributed cache entry that has
been previously localized to a disk that is now considered full/bad, the dist cache directory
will _not_ be bind-mounted into the container's filesystem namespace.  At that point any symlinks
in the container's current working directory that point to those disks will reference invalid

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message