hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wilfred Spiegelenburg (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2902) Killing a container that is localizing can orphan resources in the DOWNLOADING state
Date Wed, 04 Jan 2017 01:22:59 GMT

    [ https://issues.apache.org/jira/browse/YARN-2902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15796805#comment-15796805
] 

Wilfred Spiegelenburg commented on YARN-2902:
---------------------------------------------

I was looking at a problem around removing non existing directories in the LCE (native code)
and saw that there was a difference in behaviour between trunk and branch-2.
In trunk we "ignore" a non existing directory in {{delete_as_user()}} whan the stat returns
an {{ENOENT}} we do not do that in branch-2. I tracked it back to the backport of this jira
into branch-2. The native code part of the change was not ported back into branch-2.

Question is was that on purpose or was it an oversight? If it was an oversight does this require
a new jira to backport the native code or does it get handled as an addendum to this one.
It has been a while since this jira was closed and the jira has been in a number of releases.

> Killing a container that is localizing can orphan resources in the DOWNLOADING state
> ------------------------------------------------------------------------------------
>
>                 Key: YARN-2902
>                 URL: https://issues.apache.org/jira/browse/YARN-2902
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager
>    Affects Versions: 2.5.0
>            Reporter: Jason Lowe
>            Assignee: Varun Saxena
>             Fix For: 2.8.0, 2.7.2, 2.6.4, 3.0.0-alpha1
>
>         Attachments: YARN-2902-branch-2.6.01.patch, YARN-2902.002.patch, YARN-2902.03.patch,
YARN-2902.04.patch, YARN-2902.05.patch, YARN-2902.06.patch, YARN-2902.07.patch, YARN-2902.08.patch,
YARN-2902.09.patch, YARN-2902.10.patch, YARN-2902.11.patch, YARN-2902.patch
>
>
> If a container is in the process of localizing when it is stopped/killed then resources
are left in the DOWNLOADING state.  If no other container comes along and requests these resources
they linger around with no reference counts but aren't cleaned up during normal cache cleanup
scans since it will never delete resources in the DOWNLOADING state even if their reference
count is zero.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message