hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3400) Facilitate creation of temporary files in HDFS
Date Tue, 20 May 2008 16:29:55 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12598349#action_12598349
] 

Steve Loughran commented on HADOOP-3400:
----------------------------------------

While useful, this code is built on the assumptions that GC/shutdown happens cleanly,. and
that when it does, the filesystem is visible. A long-lived filesystem will still pick up lots
of orphan temp files.

What may be (eventually) useful would be the ability to give files an expiry time in their
metadata, and have something intermittently walk the FS, deleting files that have expired.




> Facilitate creation of temporary files in HDFS
> ----------------------------------------------
>
>                 Key: HADOOP-3400
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3400
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>         Attachments: deleteOnExit.patch, deleteOnExit.patch, deleteOnExit.patch
>
>
> There are a set of applications that use HDFS to create temporary files. The application
would ideally like these files to be automatically deleted when the application process exits.
This is similar to the File.deleteOnExit() in the Java API.
> One proposal is to add a new method in FileSystem
> public void deleteOnExit(Path)
>     This API requests that the file or directory denoted by this abstract pathname be
deleted when the virtual machine terminates. Deletion will be attempted only for normal termination
of the virtual machine, as defined by the Java Language Specification. Once deletion has been
requested, it is not possible to cancel the request. This method should therefore be used
with care.
> This method can be implemented entirely in the client side code, e.g. FileSystem.java
will keep a cache of all the pathnames specified by the above API. FileSystem.close will invoke
delete() on all the pathnames found in the cache. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message