hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-2496) Changes for capacity scheduler to support allocate resource respect labels
Date Wed, 15 Oct 2014 00:21:33 GMT

     [ https://issues.apache.org/jira/browse/YARN-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Wangda Tan updated YARN-2496:
    Attachment: YARN-2496-20141014-1.patch

[~vinodkv], thanks for your comments. They are all make sense to me and addressed in the new
patch, except a few,

bq. SchedulerUtils.checkAndThrowIfLabelNotIncluded -> checkIfLabelInTheCollection()
Changed to checkIfLabelInClusterNodeLabels

bq. Seems like removing accessible labels from a queue doesn't work given the changes in AbstractCSQueue.
Removing accessible labels from a queue should be done by first modifying capacity scheduler
configuration, then call refreshQueue by RMAdminCLI

bq. I think reservations are still not handled per accessible node-labels in the patch. We
can fix it separately though.
Reservation is partially works. Resource will be reserved only when the node satisfies queue
accessible-label and the node satisfies node-label-expression of the resource request. 
The reservation will not work with the changes of YARN-1769 (continuous reservation looking).
Which is required to count reserved resource by node label, we haven't counted reserved resource
by node label.

> Changes for capacity scheduler to support allocate resource respect labels
> --------------------------------------------------------------------------
>                 Key: YARN-2496
>                 URL: https://issues.apache.org/jira/browse/YARN-2496
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: YARN-2496-20141009-1.patch, YARN-2496-20141014-1.patch, YARN-2496.patch,
YARN-2496.patch, YARN-2496.patch, YARN-2496.patch, YARN-2496.patch, YARN-2496.patch, YARN-2496.patch,
> This JIRA Includes:
> - Add/parse labels option to {{capacity-scheduler.xml}} similar to other options of queue
like capacity/maximum-capacity, etc.
> - Include a "default-label-expression" option in queue config, if an app doesn't specify
label-expression, "default-label-expression" of queue will be used.
> - Check if labels can be accessed by the queue when submit an app with labels-expression
to queue or update ResourceRequest with label-expression
> - Check labels on NM when trying to allocate ResourceRequest on the NM with label-expression
> - Respect  labels when calculate headroom/user-limit

This message was sent by Atlassian JIRA

View raw message