hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sunil G (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6599) Support rich placement constraints in scheduler
Date Thu, 04 Jan 2018 12:47:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-6599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16311298#comment-16311298

Sunil G commented on YARN-6599:

Thanks [~leftnoteasy] for working on this.

Some comments/doubts from myside:

# I think {{PlacementTags}} could be part of a util class. We are trying to fetch app id from
tags hence may be it could be renamed are placement util class or similar.
# {{AYS#normalizeSchedulingRequests}} could be moved to CS for now as we are supporting this
only in CS currently.
# {{AppSchedulingInfo#addSchedulingRequests}} could avoid second for loop. I think {{Map<SchedulerRequestKey,
SchedulingRequest> schedulingRequestMap}} is not needed. If we iterate on {{List<SchedulingRequest>
schedulingRequests}}, *SchedulerRequestKey* could created for each iteration and use it internally.
# Why are we throwing exception in SingleConstraintAppPlacementAllocator #updatePendingAsk
when requests are not NULL.
# Also i really didnt understand why we need {{private SchedulingRequest schedulingRequest
= null;}} in SingleConstraintAppPlacementAllocator. This should be for all type of allocators,
correct?. So do we need an abstract class to hold such common functionalities.
# In {{internalUpdatePendingAsk}}, could we check {{if (!schedulingRequest.equals(newSchedulingRequest))}}
earlier in method.
# Is this correct? we set sizing multiple times here.
151	      sizing.setNumAllocations(existingNumAllocations);
153	      // Compare two objects
154	      if (!schedulingRequest.equals(newSchedulingRequest)) {
155	        // Rollback #numAllocations
156	        sizing.setNumAllocations(newNumAllocations);
166	      }
168	      // Rollback #numAllocations
169	      sizing.setNumAllocations(newNumAllocations);
# *validateAndSetSchedulingRequest*  for loop has a continue statement. May not be needed.
# I think we can try to have a common evaluator to process PlacementConstraint.TargetExpression
based on each TargetType. For any target type, we are trying to get *nodePartition* and *targetAllocationTags*.
A typed evaluator could process and evaluate it based on TargetType.
# Are we supporting a case where app is changing partition for a given resource request? Or
will be considered as a new requests and old one ll be cleared.
# For TargetType.ALLOCATION_TAG processing in validateAndSetSchedulingRequest, I think we
can directly reject if any tag is not starting with {{tag.startsWith(APP_ID_TAG_PREFIX)}}
# Could we get appId from appInfo itself? 
379	      ApplicationId appId = PlacementTags.extractApplicationIdFromTag(
380	          targetTag);

> Support rich placement constraints in scheduler
> -----------------------------------------------
>                 Key: YARN-6599
>                 URL: https://issues.apache.org/jira/browse/YARN-6599
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: YARN-6599-YARN-6592.003.patch, YARN-6599-YARN-6592.004.patch, YARN-6599-YARN-6592.005.patch,
YARN-6599-YARN-6592.006.patch, YARN-6599-YARN-6592.wip.002.patch, YARN-6599.poc.001.patch

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message