hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vivek Ratan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4053) Schedulers need to know when a job has completed
Date Tue, 02 Sep 2008 04:14:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12627573#action_12627573

Vivek Ratan commented on HADOOP-4053:

A job is marked complete in JobTracker.finalizeJob(). At this point, a JobInProgressListener
listener should be notified of this change. I can see at least two options to do this: 
1. Call JobInProgressListener.jobUpdated() at this point. Ideally, this method should include
two arguments/objects: one capturing the old state and one the new state, otherwise the listener
has to figure out how the job has been updated. 
2. Add a new method to JobInProgressListener called jobCompleted(), and call that from finalizeJob().

The second is cleaner, IMO, as a job's completion is a distinct, well understood event in
the system. 

> Schedulers need to know when a job has completed
> ------------------------------------------------
>                 Key: HADOOP-4053
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4053
>             Project: Hadoop Core
>          Issue Type: Improvement
>            Reporter: Vivek Ratan
> The JobInProgressListener interface is used by the framework to notify Schedulers of
when jobs are added, removed, or updated. Right now, there is no way for the Scheduler to
know that a job has completed. jobRemoved() is called when a job is retired, which can happen
many hours after a job is actually completed. jobUpdated() is called when a job's priority
is changed. We need to notify a listener when a job has completed (either successfully, or
has failed or been killed). 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message