hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mathias Herberts <mathias.herbe...@gmail.com>
Subject Re: Maintaining map reduce job logs - The best practices
Date Fri, 23 Sep 2011 13:13:39 GMT
> You can find the job specific logs in two places. The first one is in the hdfs ouput directory.
The second place is under $HADOOP_HOME/logs/history ($HADOOP_HOME/logs/history/done)
> Both these paces have the config file and the job logs for each submited job.

Those logs in 'history/done' will get discarded after an
unconfigurable (as of yet) delay set to 30 days IIRC.

Our strategy is to move those logs daily in 'history/archive' which
won't get wiped.

View raw message