hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun Suresh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4888) Changes in RM AppSchedulingInfo for identifying resource-requests explicitly
Date Thu, 14 Jul 2016 06:11:21 GMT

    [ https://issues.apache.org/jira/browse/YARN-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376380#comment-15376380

Arun Suresh commented on YARN-4888:

Thanks for the patch [~subru]

Couple of initial comments:
# In {{AppSchedulingInfo::checkForDeactivation()}}, given that you are introducing an inner
for loop, the deactivate = false should be set only if ALL requests in the {{mappedRequest.values()}}
has {{request.getNumContainers() > 0}} right ?
# In {{AppSchedulingInfo::allocateNodeLocal()}}, That {{rackLocalRequest}} and {{offRackRequest}}
selected from the {{resoureRequestMap}} should correspond to the same allocationRequestId
of the {{nodeLocalRequest}}. Only when you have a single allocationRequestId, can you guarantee
that {{.firstEntry().getValue()}} will correspond to the same nodeLocalRequest in both cases.

With regard to the getMergeResource() method. I have a feeling we should not be merging all
requests of a {{Priority}}. Consider this alternate approach, which I feel might allow us
to more easily verify if we are breaking any invariants in the Scheduler:
My intuition is based on the fact that If we agree that, in the absence of an *allocateRequestId*,
we can simulate the same functionality by using a unique Priority to tie all requests we need
to be of the same allocateRequestId....  Then, why not replace the 'Priority' in the {{AppSchedulingInfo::resourceRequestMap}}
with a new type (called *SchedulerPriority*) which is essentially a composite of *Priority
+ allocateRequestId*. If not allocateRequestId is provided, it will default to 0. If the *SchedulerPriority*
is a subclass of *Priority*, then we wont even need to change any of the APIs.

Thoughts ?
I can help provide a quick prototype patch to verify if this works..

> Changes in RM AppSchedulingInfo for identifying resource-requests explicitly
> ----------------------------------------------------------------------------
>                 Key: YARN-4888
>                 URL: https://issues.apache.org/jira/browse/YARN-4888
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Subru Krishnan
>            Assignee: Subru Krishnan
>         Attachments: YARN-4888-v0.patch
> YARN-4879 puts forward the notion of identifying allocate requests explicitly. This JIRA
is to track the changes in RM app scheduling data structures to accomplish it. Please refer
to the design doc in the parent JIRA for details.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message