hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anubhav Dhoot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2005) Blacklisting support for scheduling AMs
Date Mon, 29 Jun 2015 22:29:06 GMT

    [ https://issues.apache.org/jira/browse/YARN-2005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606549#comment-14606549
] 

Anubhav Dhoot commented on YARN-2005:
-------------------------------------

The getNumActiveNMs gets number of unique NodeId, while blacklisting occurs on the host part
of the NodeId. This ensures we calculate 2 different NodeId on the same host as 1 host for
blacklisting. This ends up exposed in the unit tests where were use multiple NMs on same host.

I was wondering about how often to update this. Should we do it every time we try to get blacklist
(add another api on BlacklistManager) or during start of each attempt? I am thinking every
time we get blacklist should be ideal. I will update a patch with that.

> Blacklisting support for scheduling AMs
> ---------------------------------------
>
>                 Key: YARN-2005
>                 URL: https://issues.apache.org/jira/browse/YARN-2005
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: resourcemanager
>    Affects Versions: 0.23.10, 2.4.0
>            Reporter: Jason Lowe
>            Assignee: Anubhav Dhoot
>         Attachments: YARN-2005.001.patch, YARN-2005.002.patch
>
>
> It would be nice if the RM supported blacklisting a node for an AM launch after the same
node fails a configurable number of AM attempts.  This would be similar to the blacklisting
support for scheduling task attempts in the MapReduce AM but for scheduling AM attempts on
the RM side.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message