Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Fri, 4 Nov 2016 18:45:58 +0000 (UTC)
From: "Daniel Templeton (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.13006051.1474322132000.193445.1478285158904@Atlassian.JIRA>
In-Reply-To: <JIRA.13006051.1474322132000@Atlassian.JIRA>
References: <JIRA.13006051.1474322132000@Atlassian.JIRA> <JIRA.13006051.1474322132893@arcas>
Subject: [jira] [Commented] (YARN-5658) YARN should have a hook to delete a
 path from HDFS when an application ends
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Fri, 04 Nov 2016 18:46:00 -0000


    [ https://issues.apache.org/jira/browse/YARN-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15637299#comment-15637299 ] 

Daniel Templeton commented on YARN-5658:
----------------------------------------

YARN-2261 isn't itself sufficient.  It provides the hook to run arbitrary code, but unless all those AMs are extended to use YARN-2261 to offer a mechanism to delete a directory, it doesn't help me here.

Using YARN-2261 here seems like overkill.  I want to delete a directory in HDFS, so have to allocate a container and run a script to launch another JVM to talk to HDFS to delete the directory.

> YARN should have a hook to delete a path from HDFS when an application ends
> ---------------------------------------------------------------------------
>
>                 Key: YARN-5658
>                 URL: https://issues.apache.org/jira/browse/YARN-5658
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: resourcemanager
>            Reporter: Daniel Templeton
>            Assignee: Daniel Templeton
>
> There are many cases when a client uploads data to HDFS and then needs to subsequently clean it up, such as with the distributed cache.  It would be helpful if YARN would do that cleanup automatically on job completion.
> The hook could be generic to an URI supported by {{FileSystem}}.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org