hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4465) SchedulerUtils#validateRequest for Label check should happen only when nodelabel enabled
Date Thu, 28 Jan 2016 04:21:39 GMT

    [ https://issues.apache.org/jira/browse/YARN-4465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15120726#comment-15120726

Wangda Tan commented on YARN-4465:

Hi [~bibinchundatt],

I felt I made a mistake for previous comment, when the label is not enabled, we shouldn't
simply ignore checking labels in request. Request with label will be starving when labels
are disabled. Reset such labels to empty is also problematic, we should let application knows
about the failure.

Here is my proposal, when label is disabled:
- Any request with label will be rejected (throws invalid label exception)
- Queues with label configured will be rejected, including default-node-label-expression,
accessible-node-labels (accessible-node-labels is * by default, but we shouldn't allow explicitly
set accessible-node-labels to non-empty and non *), and label-related capacities (check {{org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CSQueueUtils#loadCapacitiesByLabelsFromConf}})

Queue related changes could be done in a separated JIRA.


> SchedulerUtils#validateRequest for Label check should happen only when nodelabel enabled
> ----------------------------------------------------------------------------------------
>                 Key: YARN-4465
>                 URL: https://issues.apache.org/jira/browse/YARN-4465
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Bibin A Chundatt
>            Assignee: Bibin A Chundatt
>            Priority: Minor
>         Attachments: 0001-YARN-4465.patch, 0002-YARN-4465.patch, 0003-YARN-4465.patch
> Disable label from rm side yarn.nodelabel.enable=false
> Capacity scheduler label configuration for queue is available as below
> default label for queue = b1 as 3 and accessible labels as 1,3
> Submit application to queue A .
> {noformat}
> Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException):
Invalid resource request, queue=b1 doesn't have permission to access all labels in resource
request. labelExpression of resource request=3. Queue labels=1,3
>         at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils.validateResourceRequest(SchedulerUtils.java:304)
>         at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils.normalizeAndValidateRequest(SchedulerUtils.java:234)
>         at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils.normalizeAndValidateRequest(SchedulerUtils.java:216)
>         at org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.validateAndCreateResourceRequest(RMAppManager.java:401)
>         at org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.createAndPopulateNewRMApp(RMAppManager.java:340)
>         at org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.submitApplication(RMAppManager.java:283)
>         at org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitApplication(ClientRMService.java:602)
>         at org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitApplication(ApplicationClientProtocolPBServiceImpl.java:247)
> {noformat}
> # Ignore default label expression when label is disabled *or*
> # NormalizeResourceRequest we can set label expression to <DEFAULT PARTITION> when
node label is not enabled *or*
> # Improve message

This message was sent by Atlassian JIRA

View raw message