hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2473) YARN never cleans up container directories from a full disk
Date Fri, 29 Aug 2014 17:53:54 GMT

    [ https://issues.apache.org/jira/browse/YARN-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14115580#comment-14115580

Jason Lowe commented on YARN-2473:

No worries, Varun, we all missed it. ;-)

We may need to track full disks separately from "bad" disks so we can know whether or not
it's OK to try to delete a container directory from a particular disk that isn't a known "good"
disk.  I'm hesitant to have the NM try to remove container directories even from "bad" disks
since touching them can cause a very long pause for the thread that did it.

> YARN never cleans up container directories from a full disk
> -----------------------------------------------------------
>                 Key: YARN-2473
>                 URL: https://issues.apache.org/jira/browse/YARN-2473
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>    Affects Versions: 2.4.0
>            Reporter: Jason Lowe
>            Priority: Blocker
> After YARN-1781 when a container ends up filling a local disk the nodemanager will mark
it as a bad disk and remove it from the list of good local dirs.  When the container eventually
completes the files that filled the disk will not be removed because the NM thinks the directory
is bad.

This message was sent by Atlassian JIRA

View raw message