From yarn-issues-return-137061-archive-asf-public=cust-asf.ponee.io@hadoop.apache.org Fri Feb 2 09:46:07 2018 Return-Path: X-Original-To: archive-asf-public@eu.ponee.io Delivered-To: archive-asf-public@eu.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by mx-eu-01.ponee.io (Postfix) with ESMTP id 2E03D180608 for ; Fri, 2 Feb 2018 09:46:07 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 1DA58160C57; Fri, 2 Feb 2018 08:46:07 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 63771160C41 for ; Fri, 2 Feb 2018 09:46:06 +0100 (CET) Received: (qmail 22085 invoked by uid 500); 2 Feb 2018 08:46:05 -0000 Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list yarn-issues@hadoop.apache.org Received: (qmail 22065 invoked by uid 99); 2 Feb 2018 08:46:05 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 02 Feb 2018 08:46:05 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id F187DC0505 for ; Fri, 2 Feb 2018 08:46:04 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -109.511 X-Spam-Level: X-Spam-Status: No, score=-109.511 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id 0WeylYpJhnrL for ; Fri, 2 Feb 2018 08:46:03 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id CF04B5FB58 for ; Fri, 2 Feb 2018 08:46:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 1450CE02F7 for ; Fri, 2 Feb 2018 08:46:01 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 8551C21E87 for ; Fri, 2 Feb 2018 08:46:00 +0000 (UTC) Date: Fri, 2 Feb 2018 08:46:00 +0000 (UTC) From: "Weiwei Yang (JIRA)" To: yarn-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (YARN-7757) Refactor NodeLabelsProvider to be more generic and reusable for node attributes providers MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/YARN-7757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349983#comment-16349983 ] Weiwei Yang commented on YARN-7757: ----------------------------------- Per offline discussion with [~Naganarasimha], uploaded v6 patch majorly introduced another abstract class layer {{NodeLabelsProvider}} and {{NodeAttributesProvider}}, to avoid a potential typing mis-match while initializing by reflection. Other improvements [~Naganarasimha] mentioned we agree to postpone to individual jiras so we can get this blocker done first. Some details for reference: bq. It looks like it can create only one of the provider, either for labels or for attributes. I think we need to explicitly support for both. This will be done in YARN-7871 bq. multi scripts for different types of attributes Our configuration doesn't allow to configure multiple scripts now, it will fail on script verification. Right now we do not see a need to support this, but we can revisit if necessary. bq. Comments over NodeManager and NodeStatusUpdate Addressed in v6 patch. bq. verifyConfiguredScript seems to be out of place Right now the verifyConfiguredScript is only used by scripted based providers, lets keep it for now. If further we see it can be reused in some place else, we can pull it out. bq. serviceStart needs to capture that taskInterval needs to be set before the service is started It is initiated with -1 value, and gets override by particular provider. bq. Lets use scheduledexecutorservice instead of timer task ... We have agreed on this, but since this is not a work of refactoring, we agreed to open another lower priority JIRA to track. bq. output format of ScriptBasedNodeAttributesProvider This will need to be taken care of by YARN-7871 once we decided the finalized format of the attributes and conventions. This also depends on YARN-7856. Hope this addresses everything so far. Thanks. > Refactor NodeLabelsProvider to be more generic and reusable for node attributes providers > ----------------------------------------------------------------------------------------- > > Key: YARN-7757 > URL: https://issues.apache.org/jira/browse/YARN-7757 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager > Reporter: Weiwei Yang > Assignee: Weiwei Yang > Priority: Blocker > Attachments: YARN-7757-YARN-3409.001.patch, YARN-7757-YARN-3409.002.patch, YARN-7757-YARN-3409.003.patch, YARN-7757-YARN-3409.004.patch, YARN-7757-YARN-3409.005.patch, YARN-7757-YARN-3409.006.patch, nodeLabelsProvider_refactor_class_hierarchy.pdf, nodeLabelsProvider_refactor_v2.pdf > > > Propose to do refactor on {{NodeLabelsProvider}}, {{AbstractNodeLabelsProvider}} to be more generic, so node attributes providers can reuse these interface/abstract classes. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org For additional commands, e-mail: yarn-issues-help@hadoop.apache.org