From core-dev-return-51863-apmail-hadoop-core-dev-archive=hadoop.apache.org@hadoop.apache.org Tue Oct 07 14:04:37 2008 Return-Path: Delivered-To: apmail-hadoop-core-dev-archive@www.apache.org Received: (qmail 19808 invoked from network); 7 Oct 2008 14:04:37 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 7 Oct 2008 14:04:37 -0000 Received: (qmail 3448 invoked by uid 500); 7 Oct 2008 14:04:33 -0000 Delivered-To: apmail-hadoop-core-dev-archive@hadoop.apache.org Received: (qmail 3422 invoked by uid 500); 7 Oct 2008 14:04:33 -0000 Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: core-dev@hadoop.apache.org Delivered-To: mailing list core-dev@hadoop.apache.org Received: (qmail 3411 invoked by uid 99); 7 Oct 2008 14:04:33 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 07 Oct 2008 07:04:33 -0700 X-ASF-Spam-Status: No, hits=-1999.9 required=10.0 tests=ALL_TRUSTED,DNS_FROM_SECURITYSAGE X-Spam-Check-By: apache.org Received: from [140.211.11.140] (HELO brutus.apache.org) (140.211.11.140) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 07 Oct 2008 14:03:38 +0000 Received: from brutus (localhost [127.0.0.1]) by brutus.apache.org (Postfix) with ESMTP id 52418234C211 for ; Tue, 7 Oct 2008 07:03:44 -0700 (PDT) Message-ID: <609810383.1223388224335.JavaMail.jira@brutus> Date: Tue, 7 Oct 2008 07:03:44 -0700 (PDT) From: "Amar Kamat (JIRA)" To: core-dev@hadoop.apache.org Subject: [jira] Updated: (HADOOP-4053) Schedulers need to know when a job has completed In-Reply-To: <156825368.1220328524223.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HADOOP-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amar Kamat updated HADOOP-4053: ------------------------------- Attachment: HADOOP-4053-v4.patch Attaching a patch the incorporates Hemanth's and Vivek's comments. bq. JobChangeEvent: {{JobChangeEvent}} clearly reflects that the job in hadoop has changed. So any external entity that needs to monitor hadoop jobs should expect {{JobChangeEvent}}. I think for now we can keep it simple and extend {{JobChangeEvent}} if required in future. bq. JobStatusChangeEvent: +1 bq. Other JobInProgressListener sub-classes: +1 bq. JobQueuesManager: +1 bq. JobTracker: Considering Vivek's comments we would probably need to call {{jobUpdated()}} twice. bq. Tests: +1 bq. What does RUN_STATE mean? _RUN_STATE_ is used throughout the framework to indicate the state of the object. It has nothing to do with the object being in the _RUNNING_ state. In order to be consistent, I think we should keep it _RUN_STATE_. I think appending the events with _CHANGED_ looks ok to me. bq. I don't feel very comfortable with the fact that JobStatusChangeEvent can contain multiple Events? +1. Will make it 1-1 mapping from {{JobStatusChangeEvent}} object to {{EventType}}. _test patch_ and _ant test_ passed on my box. > Schedulers need to know when a job has completed > ------------------------------------------------ > > Key: HADOOP-4053 > URL: https://issues.apache.org/jira/browse/HADOOP-4053 > Project: Hadoop Core > Issue Type: Improvement > Affects Versions: 0.19.0 > Reporter: Vivek Ratan > Assignee: Amar Kamat > Priority: Blocker > Attachments: HADOOP-4053-v1.patch, HADOOP-4053-v2.patch, HADOOP-4053-v3.1.patch, HADOOP-4053-v3.2.patch, HADOOP-4053-v4.patch > > > The JobInProgressListener interface is used by the framework to notify Schedulers of when jobs are added, removed, or updated. Right now, there is no way for the Scheduler to know that a job has completed. jobRemoved() is called when a job is retired, which can happen many hours after a job is actually completed. jobUpdated() is called when a job's priority is changed. We need to notify a listener when a job has completed (either successfully, or has failed or been killed). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.