hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2496) Changes for capacity scheduler to support allocate resource respect labels
Date Tue, 14 Oct 2014 01:47:34 GMT

    [ https://issues.apache.org/jira/browse/YARN-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14170360#comment-14170360
] 

Vinod Kumar Vavilapalli commented on YARN-2496:
-----------------------------------------------

Started looking at the patch. Some early comments

CapacitySchedulerConfiguration
{code} 
     LOG.debug("CSConf - getCapacityOfLabel: prefix="
          + getNodeLabelPrefix(queue, label) + ", capacity=" + capacity);
{code}
Wrap in LOG.isDebugEnabled()

Rename continueLooking variable in setupQueueConfigs to refer to reservation

LeafQueue.getRequestLabelSetByExpression() can be a static method.

AbstractQueue
 - absoluteNodeLabelCapacities -> absoluteCapacitiesByNodeLabels
 - nodeLabelCapacities -> capacitiesByNodeLabels
 - absoluteMaximumNodeLabelCapacities -> absoluteMaxCapacityByNodeLabels
 - maximumNodeLabelCapacities -> maxCapacitiesByNodeLabels
 - Similarly the CSQueueUtils.compute* methods
 - labels -> accessibleLabels
 - releaseResource(): nodeLabels.isEmpty() should also do a null check in case admin removed
accessible labels for this queue?
 - getCapacityByNodeLabel(): If parent is null, simply return 1; Similarly no need to check
!nodeLabelCapacities.containsKey(label) 
 - Same as above in getAbsoluteCapacityByNodeLabel(). Also, for root, it is set to 1 incorrectly.

SchedulerUtils.checkAndThrowIfLabelNotIncluded -> checkIfLabelInTheCollection()

Seems like removing accessible labels from a queue doesn't work given the changes in AbstractCSQueue.

> Changes for capacity scheduler to support allocate resource respect labels
> --------------------------------------------------------------------------
>
>                 Key: YARN-2496
>                 URL: https://issues.apache.org/jira/browse/YARN-2496
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: YARN-2496-20141009-1.patch, YARN-2496.patch, YARN-2496.patch, YARN-2496.patch,
YARN-2496.patch, YARN-2496.patch, YARN-2496.patch, YARN-2496.patch, YARN-2496.patch
>
>
> This JIRA Includes:
> - Add/parse labels option to {{capacity-scheduler.xml}} similar to other options of queue
like capacity/maximum-capacity, etc.
> - Include a "default-label-expression" option in queue config, if an app doesn't specify
label-expression, "default-label-expression" of queue will be used.
> - Check if labels can be accessed by the queue when submit an app with labels-expression
to queue or update ResourceRequest with label-expression
> - Check labels on NM when trying to allocate ResourceRequest on the NM with label-expression
> - Respect  labels when calculate headroom/user-limit



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message