hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From MARCOS MEDRADO RUBINELLI <marc...@buscapecompany.com>
Subject Re: Job log location and retention
Date Wed, 03 Apr 2013 12:06:25 GMT

The jobtracker doesn't keep a reference to the job to save memory, but you may still find
it in the filesystem. For a default CDH3 installation, it will be in the jobtracker's local
filesystem, at /var/log/hadoop-0.20/history/done/

Logs from individual tasks are a little trickier to find. They will be in the node that ran
that task, at /var/log/hadoop-0.20/history/userlogs if they haven't been deleted yet -- deletion
is controled by mapreduce.job.userlog.retain.hours, and the default is 24 hours.


I would like to ask why the logs (configurations and status) of my job disappeared in the
Jobtracker web UI?

Specifically, I finished a job four days ago, but I cannot find it in the Jobtracker web UI,
neither in the
homepage nor "Job Tracker History" on the left-bottom corner.

The cluster is deployed with CDH3, and users actively submit jobs.
Some configurations:
mapred.jobtracker.retirejob.interval    3600000
mapred.jobtracker.completeuserjobs.maximum      5

View raw message