hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anubhav Dhoot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2005) Blacklisting support for scheduling AMs
Date Mon, 29 Jun 2015 22:29:06 GMT

    [ https://issues.apache.org/jira/browse/YARN-2005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606549#comment-14606549

Anubhav Dhoot commented on YARN-2005:

The getNumActiveNMs gets number of unique NodeId, while blacklisting occurs on the host part
of the NodeId. This ensures we calculate 2 different NodeId on the same host as 1 host for
blacklisting. This ends up exposed in the unit tests where were use multiple NMs on same host.

I was wondering about how often to update this. Should we do it every time we try to get blacklist
(add another api on BlacklistManager) or during start of each attempt? I am thinking every
time we get blacklist should be ideal. I will update a patch with that.

> Blacklisting support for scheduling AMs
> ---------------------------------------
>                 Key: YARN-2005
>                 URL: https://issues.apache.org/jira/browse/YARN-2005
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: resourcemanager
>    Affects Versions: 0.23.10, 2.4.0
>            Reporter: Jason Lowe
>            Assignee: Anubhav Dhoot
>         Attachments: YARN-2005.001.patch, YARN-2005.002.patch
> It would be nice if the RM supported blacklisting a node for an AM launch after the same
node fails a configurable number of AM attempts.  This would be similar to the blacklisting
support for scheduling task attempts in the MapReduce AM but for scheduling AM attempts on
the RM side.

This message was sent by Atlassian JIRA

View raw message