hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhruba borthakur (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5445) HDFS Tmpreaper
Date Mon, 27 Apr 2009 23:49:30 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12703444#action_12703444
] 

dhruba borthakur commented on HADOOP-5445:
------------------------------------------

> it is really helpful for a cronjob to come along and cleanup transient results that no
one is using so that disk space can be recovered.

We have been doing precisely that. We have a cron job (outside of hadoop) to clean up left-over
hdfs files periodically. 

> HDFS Tmpreaper
> --------------
>
>                 Key: HADOOP-5445
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5445
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: dfs
>         Environment: CentOs 4/5, Java 1.5, Hadoop 0.17.3
>            Reporter: Michael Andrews
>            Priority: Minor
>             Fix For: 0.17.3
>
>         Attachments: DateDelta.java, TmpReaper.java
>
>
> Java implementation of tmpreaper utility for HDFS.  Helps when you expect processes to
die before they can clean up.  I have perl unit tests that can be ported over to java or groovy
if the hadoop team is interested in this utility.  One issue is that the unit tests set the
modification time of test files, which is unsupported in HDFS (as far as I can tell). 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message