hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amar Kamat (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-4053) Schedulers need to know when a job has completed
Date Tue, 07 Oct 2008 14:03:44 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Amar Kamat updated HADOOP-4053:

    Attachment: HADOOP-4053-v4.patch

Attaching a patch the incorporates Hemanth's and Vivek's comments.

bq. JobChangeEvent:
{{JobChangeEvent}} clearly reflects that the job in hadoop has changed. So any external entity
that needs to monitor hadoop jobs should expect {{JobChangeEvent}}. I think for now we can
keep it simple and extend {{JobChangeEvent}} if required in future. 
bq. JobStatusChangeEvent:
bq. Other JobInProgressListener sub-classes:
bq. JobQueuesManager:
bq. JobTracker:
Considering Vivek's comments we would probably need to call {{jobUpdated()}} twice.
bq. Tests:
bq. What does RUN_STATE mean?
_RUN_STATE_ is used throughout the framework to indicate the state of the object. It has nothing
to do with the object being in the _RUNNING_ state. In order to be  consistent, I think we
should keep it _RUN_STATE_. I think appending the events with _CHANGED_ looks ok to me.
bq. I don't feel very comfortable with the fact that JobStatusChangeEvent can contain multiple
+1. Will make it 1-1 mapping from {{JobStatusChangeEvent}} object to {{EventType}}.

_test patch_ and _ant test_ passed on my box.

> Schedulers need to know when a job has completed
> ------------------------------------------------
>                 Key: HADOOP-4053
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4053
>             Project: Hadoop Core
>          Issue Type: Improvement
>    Affects Versions: 0.19.0
>            Reporter: Vivek Ratan
>            Assignee: Amar Kamat
>            Priority: Blocker
>         Attachments: HADOOP-4053-v1.patch, HADOOP-4053-v2.patch, HADOOP-4053-v3.1.patch,
HADOOP-4053-v3.2.patch, HADOOP-4053-v4.patch
> The JobInProgressListener interface is used by the framework to notify Schedulers of
when jobs are added, removed, or updated. Right now, there is no way for the Scheduler to
know that a job has completed. jobRemoved() is called when a job is retired, which can happen
many hours after a job is actually completed. jobUpdated() is called when a job's priority
is changed. We need to notify a listener when a job has completed (either successfully, or
has failed or been killed). 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message