hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-239) job tracker WI drops jobs after 24 hours
Date Tue, 26 Sep 2006 18:15:51 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-239?page=comments#action_12437898 ] 
            
Doug Cutting commented on HADOOP-239:
-------------------------------------

Perhaps we should change the string constants in JobInfo.java to be an enum (or two), to get
the compiler to do a bit more work for us?  Or, we could use a record i/o record and XML output.
 This way serializers and deserializers would be written for us.  That's perhaps overkill,
but it's worth thinking about.

In any case, we need to make sure it's easy for these files to evolve.  Adding and removing
fields should be trivial, and processing old files that have obsolete or missing fields should
also be natural.  Right now, each new field requires a number of coordinated changes.  Is
there any way we can reduce these?

> job tracker WI drops jobs after 24 hours
> ----------------------------------------
>
>                 Key: HADOOP-239
>                 URL: http://issues.apache.org/jira/browse/HADOOP-239
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Yoram Arnon
>         Assigned To: Sanjay Dahiya
>            Priority: Minor
>         Attachments: Hadoop-239.patch, Hadoop-239_1.patch
>
>
> The jobtracker's WI, keeps track of jobs executed in the past 24 hours.
> if the cluster was idle for a day (say Sunday) it drops all its history.
> Monday morning, the page is empty.
> Better would be to store a fixed number of jobs (say 10 each of succeeded and failed
jobs).
> Also, if the job tracker is restarted, it loses all its history.
> The history should be persistent, withstanding restarts and upgrades.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message