hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4680) Job history cleaner should only check timestamps of files in old enough directories
Date Tue, 29 Oct 2013 00:20:31 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13807514#comment-13807514
] 

Hudson commented on MAPREDUCE-4680:
-----------------------------------

SUCCESS: Integrated in Hadoop-trunk-Commit #4663 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/4663/])
MAPREDUCE-4680. Job history cleaner should only check timestamps of files in old enough directories
(Robert Kanter via Sandy Ryza) (sandy: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1536558)
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobHistoryUtils.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/jobhistory/TestJobHistoryUtils.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryFileManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobHistory.java


> Job history cleaner should only check timestamps of files in old enough directories
> -----------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-4680
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4680
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobhistoryserver
>    Affects Versions: 2.0.0-alpha
>            Reporter: Sandy Ryza
>            Assignee: Robert Kanter
>             Fix For: 2.2.1
>
>         Attachments: MAPREDUCE-4680.patch, MAPREDUCE-4680.patch, MAPREDUCE-4680.patch,
MAPREDUCE-4680.patch, MAPREDUCE-4680.patch
>
>
> Job history files are stored in yyyy/mm/dd folders.  Currently, the job history cleaner
checks the modification date of each file in every one of these folders to see whether it's
past the maximum age.  The load on HDFS could be reduced by only checking the ages of files
in directories that are old enough, as determined by their name.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message