hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Templeton (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5658) YARN should have a hook to delete a path from HDFS when an application ends
Date Fri, 04 Nov 2016 18:45:58 GMT

    [ https://issues.apache.org/jira/browse/YARN-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15637299#comment-15637299

Daniel Templeton commented on YARN-5658:

YARN-2261 isn't itself sufficient.  It provides the hook to run arbitrary code, but unless
all those AMs are extended to use YARN-2261 to offer a mechanism to delete a directory, it
doesn't help me here.

Using YARN-2261 here seems like overkill.  I want to delete a directory in HDFS, so have to
allocate a container and run a script to launch another JVM to talk to HDFS to delete the

> YARN should have a hook to delete a path from HDFS when an application ends
> ---------------------------------------------------------------------------
>                 Key: YARN-5658
>                 URL: https://issues.apache.org/jira/browse/YARN-5658
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: resourcemanager
>            Reporter: Daniel Templeton
>            Assignee: Daniel Templeton
> There are many cases when a client uploads data to HDFS and then needs to subsequently
clean it up, such as with the distributed cache.  It would be helpful if YARN would do that
cleanup automatically on job completion.
> The hook could be generic to an URI supported by {{FileSystem}}.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message