[ https://issues.apache.org/jira/browse/YARN-6050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Kanter updated YARN-6050:
--------------------------------
Attachment: YARN-6050.012.patch
Thanks [~kasha]. The 012 patch:
# I removed all Node Label changes. It now strips out the wildcard port {{NodeId}}'s
# I didn't add a {{getNodeTracker}} method because {{ClusterNodeTracker}} has generics and
things were getting really ugly with some compiler warnings and issues with rawtypes. Instead,
I kept {{getClusterNodeIdsByResourceName}} that returns {{List<NodeId>}} as before,
but I moved it out of {{YarnScheduler}} into {{ResourceScheduler}} and instead of implementing
it in each of the Scheduler subclasses, I only implemented it in {{AbstractYarnScheduler}}.
I think this is the cleanest solution.
# I added some comments to {{RMUtils#getApplicableNodeCountForAM}} to make it easier to follow.
I've also put it on ReviewBoard if that's easier to look at: https://reviews.apache.org/r/57819/
> AMs can't be scheduled on racks or nodes
> ----------------------------------------
>
> Key: YARN-6050
> URL: https://issues.apache.org/jira/browse/YARN-6050
> Project: Hadoop YARN
> Issue Type: Bug
> Affects Versions: 2.9.0, 3.0.0-alpha2
> Reporter: Robert Kanter
> Assignee: Robert Kanter
> Attachments: YARN-6050.001.patch, YARN-6050.002.patch, YARN-6050.003.patch, YARN-6050.004.patch,
YARN-6050.005.patch, YARN-6050.006.patch, YARN-6050.007.patch, YARN-6050.008.patch, YARN-6050.009.patch,
YARN-6050.010.patch, YARN-6050.011.patch, YARN-6050.012.patch
>
>
> Yarn itself supports rack/node aware scheduling for AMs; however, there currently are
two problems:
> # To specify hard or soft rack/node requests, you have to specify more than one {{ResourceRequest}}.
For example, if you want to schedule an AM only on "rackA", you have to create two {{ResourceRequest}},
like this:
> {code}
> ResourceRequest.newInstance(PRIORITY, ANY, CAPABILITY, NUM_CONTAINERS, false);
> ResourceRequest.newInstance(PRIORITY, "rackA", CAPABILITY, NUM_CONTAINERS, true);
> {code}
> The problem is that the Yarn API doesn't actually allow you to specify more than one
{{ResourceRequest}} in the {{ApplicationSubmissionContext}}. The current behavior is to either
build one from {{getResource}} or directly from {{getAMContainerResourceRequest}}, depending
on if {{getAMContainerResourceRequest}} is null or not. We'll need to add a third method,
say {{getAMContainerResourceRequests}}, which takes a list of {{ResourceRequest}} so that
clients can specify the multiple resource requests.
> # There are some places where things are hardcoded to overwrite what the client specifies.
These are pretty straightforward to fix.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org
|