hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sandy Ryza (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-4680) Job history cleaner should only check timestamps of files in old enough directories
Date Mon, 24 Sep 2012 22:18:08 GMT
Sandy Ryza created MAPREDUCE-4680:
-------------------------------------

             Summary: Job history cleaner should only check timestamps of files in old enough
directories
                 Key: MAPREDUCE-4680
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4680
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: jobhistoryserver
    Affects Versions: 2.0.0-alpha
            Reporter: Sandy Ryza


Job history files are stored in yyyy/mm/dd folders.  Currently, the job history cleaner checks
the modification date of each file in every one of these folders to see whether it's past
the maximum age.  The load on HDFS could be reduced by only checking the ages of files in
directories that are old enough, as determined by their name.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message