hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Naganarasimha G R (Naga)" <garlanaganarasi...@huawei.com>
Subject RE: Lifetime of jhist files
Date Wed, 29 Apr 2015 03:01:58 GMT
Hi Kevin,

Could check the below configuration for the job history server
mapreduce.jobhistory.max-age-ms 604800000       Job history files older than this many milliseconds
will be deleted when the history cleaner runs. Defaults to 604800000 (1 week).

you could refer other job history configs in http://hadoop.apache.org/docs/r2.7.0/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml


From: Kevin [kevin.macksamie@gmail.com]
Sent: Wednesday, April 29, 2015 01:11
To: user@hadoop.apache.org
Subject: Lifetime of jhist files


I am running CDH5.1.3 with YARN. The one issue I am having is that the jhist files are being
deleted too quickly. I set yarn.nodemanager.delete.debug-delay-sec to seven days (in seconds,
of course), but I am noticing that the job historyserver is expiring any jhist file that is
seven days old. I thought yarn.nodemanager.delete.debug-delay-sec was just to clean up local
logs on the NodeManager; I didn't think it also affected the job history files too.

Am I right in my assumptions? Is there anyway to extend the lifetime of the job history files?

BTW, my history files are stored in /user/history/done/<year>/<month>/<day>

Any feedback would be great.


View raw message