hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amar Kamat (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4053) Schedulers need to know when a job has completed
Date Fri, 10 Oct 2008 05:43:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12638486#action_12638486
] 

Amar Kamat commented on HADOOP-4053:
------------------------------------

bq. Do we need JobStatusChangeEvent(JobInProgress jip, EventType eventType, JobStatus status)
? 
The reason for providing this api is to take care of the case where the prev state or the
next state doesnt matter. For example, {{finalizeJob()}}. In such case the end state is what
ultimately matters and  passing a _null_ is a hack. I am ok to change this to pass both _old_
and _new_ status. But the case I have mentioned earlier made sense to me. 

bq. I don't think initTasksAndReportChange should be a public API.
I wanted to keep it in SYNC with {{JobInProgress.initTasks()}}. But I think, for now, we should
keep it package private. +1

bq. In fact, it seems to me that it is better to have this wrapper API where needed.
I wanted to keep all the code close to the source (here _initTasks()_). The authority to _init_
lies with the {{JobInProgress}} and hence I thought the authority to _init_ and _report-change_
should also go to {{JobInProgress}}.

> Schedulers need to know when a job has completed
> ------------------------------------------------
>
>                 Key: HADOOP-4053
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4053
>             Project: Hadoop Core
>          Issue Type: Improvement
>    Affects Versions: 0.19.0
>            Reporter: Vivek Ratan
>            Assignee: Amar Kamat
>            Priority: Blocker
>         Attachments: HADOOP-4053-v1.patch, HADOOP-4053-v2.patch, HADOOP-4053-v3.1.patch,
HADOOP-4053-v3.2.patch, HADOOP-4053-v4.1.patch, HADOOP-4053-v4.patch
>
>
> The JobInProgressListener interface is used by the framework to notify Schedulers of
when jobs are added, removed, or updated. Right now, there is no way for the Scheduler to
know that a job has completed. jobRemoved() is called when a job is retired, which can happen
many hours after a job is actually completed. jobUpdated() is called when a job's priority
is changed. We need to notify a listener when a job has completed (either successfully, or
has failed or been killed). 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message