hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Saxena (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6148) NM node count reported to AM in Allocate Response should consider requested node label partitions.
Date Mon, 06 Mar 2017 09:39:33 GMT

    [ https://issues.apache.org/jira/browse/YARN-6148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15896978#comment-15896978

Varun Saxena commented on YARN-6148:

Offline. we (me, [~Naganarasimha] and [~bibinchundatt]) had discussed some more scenarios
which need to be handled.
So following is the plan regarding what will be done.

# We will send a label to node count map containing label to active NM count mapping for requested
# If labels are disabled, we will report cluster count as earlier and not send anything in
label to active NM count mapping. AM can consider only the cluster node count reported earlier
in such cases. This will also help us in handling the case for rolling downgrades.
# Non Exclusive labels will not be reported in the label to node count unless explicitly requested
by AM.
# Mapreduce AM would consider both map and reduce label expression separately, if they are
different, while deciding on ignoring AM blacklisting. We will check label to node count mapping
to determine the applicable node counts for both map and reduce. 
# Queue can have a default label too. This needs to be reported to AM as well. Also needs
to be reported on move. Can be handled in YARN-6209. Active NM count may depend on it and
AM currently does not know anything about default label of queue.
#  Container info sent while reporting allocated containers would also contain label on which
container was allocated on. Based on this info we will determine if the node assigned is a
default label of queue, map/reduce partition or non exclusive label. We will not count non
exclusive label towards ignoring blacklisting. 


> NM node count reported to AM in Allocate Response should consider requested node label
> --------------------------------------------------------------------------------------------------
>                 Key: YARN-6148
>                 URL: https://issues.apache.org/jira/browse/YARN-6148
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Varun Saxena
>            Assignee: Varun Saxena
>         Attachments: YARN-6148.01.patch

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message