Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-issues@hadoop.apache.org
Date: Fri, 31 Jul 2015 19:18:08 +0000 (UTC)
From: "Anubhav Dhoot (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12711471.1398867896000.342849.1438370288540@Atlassian.JIRA>
In-Reply-To: <JIRA.12711471.1398867896000@Atlassian.JIRA>
References: <JIRA.12711471.1398867896000@Atlassian.JIRA>
 <JIRA.12711471.1398867896958@arcas>
Subject: [jira] [Commented] (YARN-2005) Blacklisting support for scheduling
 AMs
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/YARN-2005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14649674#comment-14649674 ] 

Anubhav Dhoot commented on YARN-2005:
-------------------------------------

Thanks [~asuresh] for the review
1. The getNumClusterHosts is described [above|https://issues.apache.org/jira/browse/YARN-2005?focusedCommentId=14606549&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14606549]. Basically blacklist is based on hostname and this counts number of unique hostnames for all NMs. 
3.  As discussed offline this is the limitation of the current scheduler api that does not have a notion of a blacklist for RM's own use vs the user's blacklist. We could end up removing a blacklisted node that user added. 
To avoid this we would have to add an API to the scheduler to manage a separate blacklist (say call it system blacklist) that we merge with the users blacklist during allocation. 
Wonder what others think about that?

> Blacklisting support for scheduling AMs
> ---------------------------------------
>
>                 Key: YARN-2005
>                 URL: https://issues.apache.org/jira/browse/YARN-2005
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: resourcemanager
>    Affects Versions: 0.23.10, 2.4.0
>            Reporter: Jason Lowe
>            Assignee: Anubhav Dhoot
>         Attachments: YARN-2005.001.patch, YARN-2005.002.patch, YARN-2005.003.patch, YARN-2005.004.patch
>
>
> It would be nice if the RM supported blacklisting a node for an AM launch after the same node fails a configurable number of AM attempts.  This would be similar to the blacklisting support for scheduling task attempts in the MapReduce AM but for scheduling AM attempts on the RM side.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)