From yarn-issues-return-154972-archive-asf-public=cust-asf.ponee.io@hadoop.apache.org Wed Oct 10 15:38:04 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 1FE8918067A for ; Wed, 10 Oct 2018 15:38:03 +0200 (CEST) Received: (qmail 70590 invoked by uid 500); 10 Oct 2018 13:38:03 -0000 Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list yarn-issues@hadoop.apache.org Received: (qmail 70579 invoked by uid 99); 10 Oct 2018 13:38:03 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 10 Oct 2018 13:38:03 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id B9453181107 for ; Wed, 10 Oct 2018 13:38:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -110.301 X-Spam-Level: X-Spam-Status: No, score=-110.301 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id X1q-IEXQni4C for ; Wed, 10 Oct 2018 13:38:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 4D0825F470 for ; Wed, 10 Oct 2018 13:38:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 9C943E111B for ; Wed, 10 Oct 2018 13:38:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 458672486A for ; Wed, 10 Oct 2018 13:38:00 +0000 (UTC) Date: Wed, 10 Oct 2018 13:38:00 +0000 (UTC) From: "Jason Lowe (JIRA)" To: yarn-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (YARN-7018) Interface for adding extra behavior to node heartbeats MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/YARN-7018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16644967#comment-16644967 ] Jason Lowe commented on YARN-7018: ---------------------------------- Thanks for updating the patch! Interface looks good for now. This JIRA should define the initial interface to the plugin and provide a default implementation of the plugin that does nothing (i.e.: the null plugin). NodeHeartbeatPluginImpl and BaseNodeHeartBeatPluginImpl should not be part of this, IMHO. Loading the plugin should not be in a base class that implements the plugin -- that belongs in he AbstractYarnScheduler code that is responsible for loading and initializing the plugin. And once that's removed from BaseNodeHeartBeatPluginImpl I don't see a need for that class. bq. Where shall we keep the plugin? RMActiveService? or Scheduler like Wangda Tan suggested earlier? We can put it in the AbstractYarnScheduler which the latest patch essentially is doing. The reinitialize method is loading the plugin and setting it up in the RMContext. > Interface for adding extra behavior to node heartbeats > ------------------------------------------------------ > > Key: YARN-7018 > URL: https://issues.apache.org/jira/browse/YARN-7018 > Project: Hadoop YARN > Issue Type: New Feature > Components: resourcemanager > Reporter: Jason Lowe > Assignee: Jason Lowe > Priority: Major > Attachments: YARN-7018.POC.001.patch, YARN-7018.POC.002.patch, YARN-7018.POC.003.patch > > > This JIRA tracks an interface for plugging in new behavior to node heartbeat processing. Adding a formal interface for additional node heartbeat processing would allow admins to configure new functionality that is scheduler-independent without needing to replace the entire scheduler. For example, both YARN-5202 and YARN-5215 had approaches where node heartbeat processing was extended to implement new functionality that was essentially scheduler-independent and could be implemented as a plugin with this interface. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org For additional commands, e-mail: yarn-issues-help@hadoop.apache.org