hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yoram Arnon (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-239) job tracker WI drops jobs after 24 hours
Date Mon, 07 Aug 2006 19:33:18 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-239?page=comments#action_12426302 ] 
            
Yoram Arnon commented on HADOOP-239:
------------------------------------

if a job tracker fails while running a job, you'd want the failed job's information shown
in the history, so you probably want to log a job in the file as soon as it's launched, rather
than when it's completed.

There are several links that can be drilled down into from the job information - tasks, failed
tasks, task trackers etc. Take care to store all that information in the log file so we do
not lose any current functionality.

creating a file per day may be good, as it will enable a simple way of keeping a configurable
length of history.

> job tracker WI drops jobs after 24 hours
> ----------------------------------------
>
>                 Key: HADOOP-239
>                 URL: http://issues.apache.org/jira/browse/HADOOP-239
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Yoram Arnon
>         Assigned To: Sanjay Dahiya
>            Priority: Minor
>
> The jobtracker's WI, keeps track of jobs executed in the past 24 hours.
> if the cluster was idle for a day (say Sunday) it drops all its history.
> Monday morning, the page is empty.
> Better would be to store a fixed number of jobs (say 10 each of succeeded and failed
jobs).
> Also, if the job tracker is restarted, it loses all its history.
> The history should be persistent, withstanding restarts and upgrades.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message