hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3400) Facilitate creation of temporary files in HDFS
Date Tue, 20 May 2008 16:55:55 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12598358#action_12598358
] 

Doug Cutting commented on HADOOP-3400:
--------------------------------------

> What may be (eventually) useful would be the ability to give files an expiry time in
their metadata, and have something intermittently walk the FS, deleting files that have expired.

The trash mechanism already supports that.  If you move something to a directory named .Trash/0807040000
in your home directory then HDFS (if configured with a non-zero fs.trash.interval) will remove
it after the 4th of July.  The API doesn't currently provide explicit support for this, but
it would be easy to add a removeAfter(Path, Date) method.

> Facilitate creation of temporary files in HDFS
> ----------------------------------------------
>
>                 Key: HADOOP-3400
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3400
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>         Attachments: deleteOnExit.patch, deleteOnExit.patch, deleteOnExit.patch
>
>
> There are a set of applications that use HDFS to create temporary files. The application
would ideally like these files to be automatically deleted when the application process exits.
This is similar to the File.deleteOnExit() in the Java API.
> One proposal is to add a new method in FileSystem
> public void deleteOnExit(Path)
>     This API requests that the file or directory denoted by this abstract pathname be
deleted when the virtual machine terminates. Deletion will be attempted only for normal termination
of the virtual machine, as defined by the Java Language Specification. Once deletion has been
requested, it is not possible to cancel the request. This method should therefore be used
with care.
> This method can be implemented entirely in the client side code, e.g. FileSystem.java
will keep a cache of all the pathnames specified by the above API. FileSystem.close will invoke
delete() on all the pathnames found in the cache. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message