Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <AD354F56741A1B47882A625909A59C692BDD87CB@SZXEML505-MBX.china.huawei.com>
References: 
 <CAJo9sUb0gKe_hARMfJpyPWuARDh5u-s1bt7r6uVRwRF8YpJtww@mail.gmail.com>
	<AD354F56741A1B47882A625909A59C692BDD87CB@SZXEML505-MBX.china.huawei.com>
Date: Fri, 15 May 2015 00:30:28 -0400
Message-ID: 
 <CAJo9sUaiBV=wod-qk+r9GAx-szj=Dc8_i+im70cvyRC=FoEgWw@mail.gmail.com>
Subject: Re: Lifetime of jhist files
From: Kevin <kevin.macksamie@gmail.com>
To: user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=089e01184838c2a23e0516174d10

--089e01184838c2a23e0516174d10
Content-Type: text/plain; charset=UTF-8

Thanks, Naga, that worked. I didn't catch that property in the
mapred-default.xml. Sorry for such a late response.

On Tue, Apr 28, 2015 at 11:01 PM, Naganarasimha G R (Naga) <
garlanaganarasimha@huawei.com> wrote:

>  Hi Kevin,
>
>  Could check the below configuration for the job history server
>   mapreduce.jobhistory.max-age-ms 604800000 Job history files older than
> this many milliseconds will be deleted when the history cleaner runs.
> Defaults to 604800000 (1 week).
> you could refer other job history configs in* http://hadoop.apache.org/docs/r2.7.0/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml
> <http://hadoop.apache.org/docs/r2.7.0/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml>*
>
> +Naga
>
>  ------------------------------
> *From:* Kevin [kevin.macksamie@gmail.com]
> *Sent:* Wednesday, April 29, 2015 01:11
> *To:* user@hadoop.apache.org
> *Subject:* Lifetime of jhist files
>
>   Hi,
>
>  I am running CDH5.1.3 with YARN. The one issue I am having is that the
> jhist files are being deleted too quickly. I
> set yarn.nodemanager.delete.debug-delay-sec to seven days (in seconds, of
> course), but I am noticing that the job historyserver is expiring any jhist
> file that is seven days old. I thought
> yarn.nodemanager.delete.debug-delay-sec was just to clean up local logs on
> the NodeManager; I didn't think it also affected the job history files too.
>
>  Am I right in my assumptions? Is there anyway to extend the lifetime of
> the job history files?
>
>  BTW, my history files are stored in
> /user/history/done/<year>/<month>/<day>
>
>  Any feedback would be great.
>
>  Thanks,
> Kevin
>

--089e01184838c2a23e0516174d10
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Thanks, Naga, that worked. I didn&#39;t catch that propert=
y in the mapred-default.xml. Sorry for such a late response.</div><div clas=
s=3D"gmail_extra"><br><div class=3D"gmail_quote">On Tue, Apr 28, 2015 at 11=
:01 PM, Naganarasimha G R (Naga) <span dir=3D"ltr">&lt;<a href=3D"mailto:ga=
rlanaganarasimha@huawei.com" target=3D"_blank">garlanaganarasimha@huawei.co=
m</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margi=
n:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">


<div>
<div style=3D"direction:ltr;font-family:Tahoma;color:#000000;font-size:10pt=
">Hi Kevin,
<div><br>
</div>
<div>Could check the below configuration for the job history server=C2=A0</=
div>
<div>
<table border=3D"1" style=3D"font-family:&#39;Times New Roman&#39;">
<tbody>
<tr>
<td><a name=3D"14d032047b84751f_mapreduce.jobhistory.max-age-ms" style=3D"c=
olor:rgb(0,75,145)">mapreduce.jobhistory.max-age-ms</a></td>
<td>604800000</td>
<td>Job history files older than this many milliseconds will be deleted whe=
n the history cleaner runs. Defaults to 604800000 (1 week).</td>
</tr>
</tbody>
</table>
<br>
you could refer other job history configs in<i>=C2=A0<a href=3D"http://hado=
op.apache.org/docs/r2.7.0/hadoop-mapreduce-client/hadoop-mapreduce-client-c=
ore/mapred-default.xml" target=3D"_blank">http://hadoop.apache.org/docs/r2.=
7.0/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml=
</a></i><br>
<br>
+Naga</div>
<div><br>
<div style=3D"font-family:Times New Roman;color:#000000;font-size:16px">
<hr>
<div style=3D"direction:ltr"><font face=3D"Tahoma" size=3D"2" color=3D"#000=
000"><b>From:</b> Kevin [<a href=3D"mailto:kevin.macksamie@gmail.com" targe=
t=3D"_blank">kevin.macksamie@gmail.com</a>]<br>
<b>Sent:</b> Wednesday, April 29, 2015 01:11<br>
<b>To:</b> <a href=3D"mailto:user@hadoop.apache.org" target=3D"_blank">user=
@hadoop.apache.org</a><br>
<b>Subject:</b> Lifetime of jhist files<br>
</font><br>
</div><div><div class=3D"h5">
<div></div>
<div>
<div dir=3D"ltr">Hi,
<div><br>
</div>
<div>I am running CDH5.1.3 with YARN. The one issue I am having is that the=
 jhist files are being deleted too quickly. I set=C2=A0yarn.nodemanager.del=
ete.debug-delay-sec to seven days (in seconds, of course), but I am noticin=
g that the job historyserver is expiring
 any jhist file that is seven days old. I thought yarn.nodemanager.delete.d=
ebug-delay-sec was just to clean up local logs on the NodeManager; I didn&#=
39;t think it also affected the job history files too.</div>
<div><br>
</div>
<div>Am I right in my assumptions? Is there anyway to extend the lifetime o=
f the job history files?</div>
<div><br>
</div>
<div>BTW, my history files are stored in /user/history/done/&lt;year&gt;/&l=
t;month&gt;/&lt;day&gt;</div>
<div><br>
</div>
<div>Any feedback would be great.</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Kevin</div>
</div>
</div>
</div></div></div>
</div>
</div>
</div>

</blockquote></div><br></div>

--089e01184838c2a23e0516174d10--