hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Andrews (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5445) HDFS Tmpreaper
Date Wed, 11 Mar 2009 17:52:50 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12680947#action_12680947
] 

Michael Andrews commented on HADOOP-5445:
-----------------------------------------

This utility is meant to delete temporary files (typically in /tmp) that are older than a
certain time.  The user can specify which directories to search for old files in. Many unix
systems run tmpreaper as cron job to clean up /tmp so that processes don't run out of temporary
file handles. 
(http://olympus.het.brown.edu/cgi-bin/man/man2html?tmpreaper+8)

The attached code is pretty much a clone of tmpreaper except that it operates on files stored
in HDFS.  Here is the usage message:

Usage:                                                                          
tmpreaper [-help -verbose -test -force -debug] <delta> <dirs1> [<dirs2>
...]    
options                                                                         
  -help (-h)            Print this message                                         
  -verbose (-v)      Be extra verbose                                           
  -test (-t)              Don't actually remove any files                            
  -force (-f)           Override failsafe, e.g. delete files modified in the last  
                            7 days                                                     
  -debug (-d)        Print out stack traces.                                    
  delta                  Amount and a unit of time, e.g. 30s,1m,5h,5d,6M,2y         
  dirs [dirs ...]                                                               




> HDFS Tmpreaper
> --------------
>
>                 Key: HADOOP-5445
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5445
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: dfs
>         Environment: CentOs 4/5, Java 1.5, Hadoop 0.17.3
>            Reporter: Michael Andrews
>            Priority: Minor
>             Fix For: 0.17.3
>
>         Attachments: DateDelta.java, TmpReaper.java
>
>
> Java implementation of tmpreaper utility for HDFS.  Helps when you expect processes to
die before they can clean up.  I have perl unit tests that can be ported over to java or groovy
if the hadoop team is interested in this utility.  One issue is that the unit tests set the
modification time of test files, which is unsupported in HDFS (as far as I can tell). 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message