hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sanjay Dahiya (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-239) job tracker WI drops jobs after 24 hours
Date Mon, 07 Aug 2006 21:04:15 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-239?page=comments#action_12426325 ] 
            
Sanjay Dahiya commented on HADOOP-239:
--------------------------------------

Here is a first cut at the information in job history. While displaying in JSP we can show
it in different views like by hosts or by jobs  -  

 - jobid 
 - jobName
 - User
 - jobconf ( job.xml ) 
 - start time 
 - finish time
 - Status 
 - total maps 
 - total reduces
 - finished maps ( if make -k )
 - finished reduces ( if make -k )
 - Available task trackers list at job start (to find which hosts never ran any tasks, not
sure if this is available from else where ?)
 
 - maps 
 	- taskid
 		- task attempt
 			- hostname
 			- start time
 			- finish time 
 			- error

 - reduces 
 	- taskid 
 		- task attempt
 			- host name 
 			- start time
 			- finish time
 			- phases 
 				- copy (?)
 					- start time
 					- finish time
 				- sort 
 					- start time
 					- finish time
 				- reduce 
 					- start time
 					- finish time
 			- status 
 			- error 
 	

> job tracker WI drops jobs after 24 hours
> ----------------------------------------
>
>                 Key: HADOOP-239
>                 URL: http://issues.apache.org/jira/browse/HADOOP-239
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Yoram Arnon
>         Assigned To: Sanjay Dahiya
>            Priority: Minor
>
> The jobtracker's WI, keeps track of jobs executed in the past 24 hours.
> if the cluster was idle for a day (say Sunday) it drops all its history.
> Monday morning, the page is empty.
> Better would be to store a fixed number of jobs (say 10 each of succeeded and failed
jobs).
> Also, if the job tracker is restarted, it loses all its history.
> The history should be persistent, withstanding restarts and upgrades.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message