Return-Path: 
 <yarn-issues-return-49195-apmail-hadoop-yarn-issues-archive=hadoop.apache.org@hadoop.apache.org>
X-Original-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org
Delivered-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 7BE1610716
	for <apmail-hadoop-yarn-issues-archive@minotaur.apache.org>;
 Thu, 26 Feb 2015 19:43:15 +0000 (UTC)
Received: (qmail 27083 invoked by uid 500); 26 Feb 2015 19:43:07 -0000
Delivered-To: apmail-hadoop-yarn-issues-archive@hadoop.apache.org
Received: (qmail 27030 invoked by uid 500); 26 Feb 2015 19:43:07 -0000
Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:yarn-issues-help@hadoop.apache.org>
List-Unsubscribe: <mailto:yarn-issues-unsubscribe@hadoop.apache.org>
List-Post: <mailto:yarn-issues@hadoop.apache.org>
List-Id: <yarn-issues.hadoop.apache.org>
Reply-To: yarn-issues@hadoop.apache.org
Delivered-To: mailing list yarn-issues@hadoop.apache.org
Received: (qmail 27018 invoked by uid 99); 26 Feb 2015 19:43:07 -0000
Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 26 Feb 2015 19:43:07 +0000
Date: Thu, 26 Feb 2015 19:43:07 +0000 (UTC)
From: "Vinod Kumar Vavilapalli (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12766460.1420822759000.26152.1424979787231@Atlassian.JIRA>
In-Reply-To: <JIRA.12766460.1420822759000@Atlassian.JIRA>
References: <JIRA.12766460.1420822759000@Atlassian.JIRA>
 <JIRA.12766460.1420822759851@arcas>
Subject: [jira] [Commented] (YARN-3025) Provide API for retrieving
 blacklisted nodes
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394


    [ https://issues.apache.org/jira/browse/YARN-3025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14339012#comment-14339012 ] 

Vinod Kumar Vavilapalli commented on YARN-3025:
-----------------------------------------------

Coming in very late, apologies.

Some comments:
 - Echoing Bikas's first comment: Today the AMs are expected to maintain their own scheduling state. With this you are changing that - part of the scheduling state will be remembered but the remaining isn't. We should clearly draw a line somewhere, what is it?
 - [~zjshen] did a very good job of dividing the persistence concerns, but what is the guarantee that is given to the app writers? "I'll return the list of blacklisted nodes whenever I can, but shoot I died, so I can't help you much" is not going to cut it. If we want reliable notifications, we should build a protocol between AM and RM about the persistence of the blacklisted node list - too much of a complexity if you ask me. Why not leave it to the apps?
 - The blacklist information is per application-attempt, and scheduler will forget previous application-attempts today. So as I understand it, the patch doesn't work.

> Provide API for retrieving blacklisted nodes
> --------------------------------------------
>
>                 Key: YARN-3025
>                 URL: https://issues.apache.org/jira/browse/YARN-3025
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Ted Yu
>            Assignee: Ted Yu
>         Attachments: yarn-3025-v1.txt, yarn-3025-v2.txt, yarn-3025-v3.txt
>
>
> We have the following method which updates blacklist:
> {code}
>   public synchronized void updateBlacklist(List<String> blacklistAdditions,
>       List<String> blacklistRemovals) {
> {code}
> Upon AM failover, there should be an API which returns the blacklisted nodes so that the new AM can make consistent decisions.
> The new API can be:
> {code}
>   public synchronized List<String> getBlacklistedNodes()
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)