hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Saxena (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6148) NM node count reported to AM in Allocate Response should consider requested node label partitions.
Date Tue, 07 Feb 2017 11:46:42 GMT

    [ https://issues.apache.org/jira/browse/YARN-6148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15855816#comment-15855816
] 

Varun Saxena commented on YARN-6148:
------------------------------------

Yes the intention is to make behavior consistent. The nodes available to the application depends
on the partitions requested by it (if they are accessible from the queue application has been
submitted to) so number of nodes reported should also honor that. 
Moreover, NM node count is used by MapReduce AM for instance, to ignore blacklisting. It should
use the node count based on requested partitions. Because, it is possible that blacklisting
will never be ignored if total number of nodes in the requested partitions(in terms of %)
are less than configured % to ignore blacklisting. We can still give a different config value
per application but it will be better to use node count based on requested partitions to determine
when blacklisting has to be ignored. 

> NM node count reported to AM in Allocate Response should consider requested node label
partitions.
> --------------------------------------------------------------------------------------------------
>
>                 Key: YARN-6148
>                 URL: https://issues.apache.org/jira/browse/YARN-6148
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Varun Saxena
>            Assignee: Varun Saxena
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message