hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amar Kamat (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4053) Schedulers need to know when a job has completed
Date Thu, 25 Sep 2008 11:41:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12634448#action_12634448

Amar Kamat commented on HADOOP-4053:

bq. 1. I'd expect jobCompleted() to be a no-op, and have jobRemoved() get called when the
job is removed.
I would expect {{jobRemoved()}} to be a no-op and {{jobComplete()}} to do what is currently
done by {{jobRemoved()}}. For all the current schedulers, {{jobRemoved()}} and {{jobComplete()}}
are essentially the same. {{jobRemoved()}} is important to schedulers for which the presence
of a job in the memory makes a difference e.g, memory sensitive schedulers. For others the
only thing that matters is the run-state of a job and hence it makes more sense to clean up
completed jobs as soon as possible. 

> Schedulers need to know when a job has completed
> ------------------------------------------------
>                 Key: HADOOP-4053
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4053
>             Project: Hadoop Core
>          Issue Type: Improvement
>    Affects Versions: 0.19.0
>            Reporter: Vivek Ratan
>            Assignee: Amar Kamat
>             Fix For: 0.19.0
>         Attachments: HADOOP-4053-v1.patch
> The JobInProgressListener interface is used by the framework to notify Schedulers of
when jobs are added, removed, or updated. Right now, there is no way for the Scheduler to
know that a job has completed. jobRemoved() is called when a job is retired, which can happen
many hours after a job is actually completed. jobUpdated() is called when a job's priority
is changed. We need to notify a listener when a job has completed (either successfully, or
has failed or been killed). 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message