hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From zheyi rong <zheyi.r...@gmail.com>
Subject Re: Job log location and retention
Date Thu, 04 Apr 2013 10:01:08 GMT
Thank you for the answer.

However, I do not have the access to the jobtracker's local system...

Fortunately I have found a workaround -- using
$hadoop job -history [output]



Zheyi Rong

On Wed, Apr 3, 2013 at 2:06 PM, MARCOS MEDRADO RUBINELLI <
marcosm@buscapecompany.com> wrote:

>  Zheyi,
> The jobtracker doesn't keep a reference to the job to save memory, but you
> may still find it in the filesystem. For a default CDH3 installation, it
> will be in the jobtracker's local filesystem, at
> /var/log/hadoop-0.20/history/done/
> Logs from individual tasks are a little trickier to find. They will be in
> the node that ran that task, at /var/log/hadoop-0.20/history/userlogs if
> they haven't been deleted yet -- deletion is controled by
> mapreduce.job.userlog.retain.hours, and the default is 24 hours.
> Regards,
> Marcos
>  I would like to ask why the logs (configurations and status) of my job
> disappeared in the Jobtracker web UI?
>  Specifically, I finished a job four days ago, but I cannot find it in
> the Jobtracker web UI, neither in the
> homepage nor "Job Tracker History" on the left-bottom corner.
>  The cluster is deployed with CDH3, and users actively submit jobs.
> Some configurations:
>   *mapred.jobtracker.retirejob.interval* 3600000
>   *mapred.jobtracker.completeuserjobs.maximum* 5

View raw message