Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-issues@hadoop.apache.org
Date: Sat, 20 Sep 2014 04:02:35 +0000 (UTC)
From: "Craig Welch (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12738747.1409734526000.84773.1411185755063@Atlassian.JIRA>
In-Reply-To: <JIRA.12738747.1409734526000@Atlassian.JIRA>
References: <JIRA.12738747.1409734526000@Atlassian.JIRA>
 <JIRA.12738747.1409734526952@arcas>
Subject: [jira] [Commented] (YARN-2496) [YARN-796] Changes for capacity
 scheduler to support allocate resource respect labels
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/YARN-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14141736#comment-14141736 ] 

Craig Welch commented on YARN-2496:
-----------------------------------

ach, not finished - anyway, re 2 - a particular job may well only be able to use nodes with one label in a queue, and so if the headroom includes nodes without that label, we'll end up with another deadlock case where it spins up reducers too early and then can't complete its maps.  It is definately a valid usecase to have a queue with two lables (a and b, as in this example) and an app which is consistently requesting only one of those two labels (from submission to completion...) - perhaps only "a" nodes have the "special resource" it needs (special hardware capability, etc).  For this reason, headroom should reflect the labels in the last resource request from the application, not the queue's labels.

(-re 5, I though * could be in requests, if no, then should not be an issue.)

> [YARN-796] Changes for capacity scheduler to support allocate resource respect labels
> -------------------------------------------------------------------------------------
>
>                 Key: YARN-2496
>                 URL: https://issues.apache.org/jira/browse/YARN-2496
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: YARN-2496.patch, YARN-2496.patch, YARN-2496.patch, YARN-2496.patch
>
>
> This JIRA Includes:
> - Add/parse labels option to {{capacity-scheduler.xml}} similar to other options of queue like capacity/maximum-capacity, etc.
> - Include a "default-label-expression" option in queue config, if an app doesn't specify label-expression, "default-label-expression" of queue will be used.
> - Check if labels can be accessed by the queue when submit an app with labels-expression to queue or update ResourceRequest with label-expression
> - Check labels on NM when trying to allocate ResourceRequest on the NM with label-expression
> - Respect  labels when calculate headroom/user-limit


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)