hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sanjay Dahiya (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-239) job tracker WI drops jobs after 24 hours
Date Mon, 07 Aug 2006 21:08:15 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-239?page=comments#action_12426326 ] 
            
Sanjay Dahiya commented on HADOOP-239:
--------------------------------------

repost, lost tabs in last post - 

Here is a first cut at the information in job history. While displaying in JSP we can show
it in different views like by hosts or by jobs  -  


- jobid 
 - jobName
 - User
 - jobconf ( job.xml ) 
 - start time 
 - finish time
 - Status 
 - total maps 
 - total reduces
 - finished maps ( if make -k )
 - finished reduces ( if make -k )
 - Available task trackers list at job start (to find which hosts never ran any tasks or if
hosts added in between. not sure if this is available from else where ?)
 
 - maps 
    - taskid
        - task attempt
            - hostname
            - start time
            - finish time 
            - error

 - reduces 
    - taskid 
        - task attempt
            - host name 
            - start time
            - finish time
            - phases 
                - copy (?)
                    - start time
                    - finish time
                - sort 
                    - start time
                    - finish time
                - reduce 
                    - start time
                    - finish time
            - status 
            - error 

> job tracker WI drops jobs after 24 hours
> ----------------------------------------
>
>                 Key: HADOOP-239
>                 URL: http://issues.apache.org/jira/browse/HADOOP-239
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Yoram Arnon
>         Assigned To: Sanjay Dahiya
>            Priority: Minor
>
> The jobtracker's WI, keeps track of jobs executed in the past 24 hours.
> if the cluster was idle for a day (say Sunday) it drops all its history.
> Monday morning, the page is empty.
> Better would be to store a fixed number of jobs (say 10 each of succeeded and failed
jobs).
> Also, if the job tracker is restarted, it loses all its history.
> The history should be persistent, withstanding restarts and upgrades.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message