hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2005) Blacklisting support for scheduling AMs
Date Thu, 29 Jan 2015 14:48:35 GMT

    [ https://issues.apache.org/jira/browse/YARN-2005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14296953#comment-14296953

Jason Lowe commented on YARN-2005:

As I mentioned earlier, as a first step I think we could implement an app-specific blacklisting
approach similar to what is done by the MapReduce AM today.  We would track, per application,
the nodes that have failed an AM attempt and refuse to launch subsequent AM attempts for that
application on those nodes.  If we want to keep it really simple, we could just do literally
that.  From there we can sprinkle additional logic to make it a bit more sophisticated, e.g.:
having the blacklisting auto-disable when the percentage of blacklisted nodes compared to
the total active nodes is above some threshold and/or the app has waited some amount of time
for an AM container for the next attempt.

> Blacklisting support for scheduling AMs
> ---------------------------------------
>                 Key: YARN-2005
>                 URL: https://issues.apache.org/jira/browse/YARN-2005
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: resourcemanager
>    Affects Versions: 0.23.10, 2.4.0
>            Reporter: Jason Lowe
> It would be nice if the RM supported blacklisting a node for an AM launch after the same
node fails a configurable number of AM attempts.  This would be similar to the blacklisting
support for scheduling task attempts in the MapReduce AM but for scheduling AM attempts on
the RM side.

This message was sent by Atlassian JIRA

View raw message