hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vivek Ratan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4053) Schedulers need to know when a job has completed
Date Fri, 10 Oct 2008 16:17:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12638582#action_12638582

Vivek Ratan commented on HADOOP-4053:

bq. Do we need JobStatusChangeEvent(JobInProgress jip, EventType eventType, JobStatus status)

I also feel we don't need this. Whether the old state or new state matters or not is a decision
left to the code that consumes the event change. The creator of this event shouldn't make
that decision. I think we should ensure that there is always an old state and new state (so
'null' should not be used). _finalizeJob()_ should also provide an old and new state. If the
Capacity Scheduler chooses to ignore the old state, so be it, but some other listener may
not. So I'm basically recommending that: 
* have only one constructor in JobStatusChangeEvent, which accepts both an old state and new
* Neither of these states should be null. 
* Can these two states be the same object? I don't see why not. I can't think of a situation
where they would be, but forcing the restriction that the states be different seems unnecessary.

Seems OK to have _JonInProgress.initTasks()_ do the event notification. _initTasks()_ changes
the state of a job, so the event notification should always happen. 

> Schedulers need to know when a job has completed
> ------------------------------------------------
>                 Key: HADOOP-4053
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4053
>             Project: Hadoop Core
>          Issue Type: Improvement
>    Affects Versions: 0.19.0
>            Reporter: Vivek Ratan
>            Assignee: Amar Kamat
>            Priority: Blocker
>         Attachments: HADOOP-4053-v1.patch, HADOOP-4053-v2.patch, HADOOP-4053-v3.1.patch,
HADOOP-4053-v3.2.patch, HADOOP-4053-v4.1.patch, HADOOP-4053-v4.patch
> The JobInProgressListener interface is used by the framework to notify Schedulers of
when jobs are added, removed, or updated. Right now, there is no way for the Scheduler to
know that a job has completed. jobRemoved() is called when a job is retired, which can happen
many hours after a job is actually completed. jobUpdated() is called when a job's priority
is changed. We need to notify a listener when a job has completed (either successfully, or
has failed or been killed). 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message