hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amar Kamat (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-721) When "mapred.job.tracker.persist.jobstatus.hours" is set to x hours, the files are not getting deleted after that x hours.It gets deleted after some random hours which is greater than x hours.
Date Wed, 08 Jul 2009 09:22:14 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12728581#action_12728581
] 

Amar Kamat commented on MAPREDUCE-721:
--------------------------------------

So here is what might have happened. Consider the following sequence of events
||Time||Event||
|x|delete threads runs once|
|x + 20| job completes|
|x + 60 | delete thread runs and finds that job has completed 40 mins so ignores it|
|x + 120| delete thread runs and deletes the job status|

So from the time the job completed, its {{(x + 120) -  (x + 20) = 1hr 40mins}}. We should
decrease the check interval. 

> When "mapred.job.tracker.persist.jobstatus.hours" is set to x hours, the files are not
getting deleted after that x hours.It gets deleted after some random hours which is greater
than x hours.
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-721
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-721
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Iyappan Srinivasan
>
> When "mapred.job.tracker.persist.jobstatus.hours" is set to 1 hour, the jobid files,
present in configured directory, are not getting deleted after that 1 hour. It gets deleted
after some random hours.
> This is irrespective of whether "mapred.job.tracker.persist.jobstatus.dir" is set to
local file system or DFS.
> Steps to reproduce the issue:
> 1) Im mapred-site.xml,
> Set "mapred.job.tracker.persist.jobstatus.active" to "true"
> Set "mapred.job.tracker.persist.jobstatus.hours" to "1"
> Set "mapred.job.tracker.persist.jobstatus.dir" to either  a local file syste by preixing
it  with file:/// or dfs by giving a dfs directory without the "file:///"
> 2) After that run any job.
> 3) After it is complete the job id file comes in the configured directory.
> 4) Wait for one hour
> 5) Observe After 1 hour the files are not deleted.
> 6) After some random hours, files gets deleted.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message