hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yufei Gu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5795) FairScheduler set AppMaster vcores didn't work
Date Thu, 24 Nov 2016 00:14:58 GMT

    [ https://issues.apache.org/jira/browse/YARN-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15691743#comment-15691743
] 

Yufei Gu commented on YARN-5795:
--------------------------------

Yes. I found this issue as well. No matter how many vcores you give it to resource request,
{{DefaultResourceCalculator}} will assign 1 to it while normalizing. 
It is hard to say if this is bug for {{DefaultResourceCalculator}} since it depends on how/when
we use it.
This is more about an issue of fair scheduler. In FairScheduler, we use the {{DominantResourceCalculator}}
in the most case, but the weird thing is if your invoke {{fairScheduler.getResourceCalculator()}},
you will get a {{DefaultResourceCalculator}}. That doesn't sound reasonable to me. We should
investigate and make some changes.
YARN-5774 can solve this issue incidentally. Maybe we could focus on the what should {{fairScheduler.getResourceCalculator()}}
return in this JIRA.

> FairScheduler set AppMaster vcores didn't work
> ----------------------------------------------
>
>                 Key: YARN-5795
>                 URL: https://issues.apache.org/jira/browse/YARN-5795
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: resourcemanager
>    Affects Versions: 2.7.1
>            Reporter: tangshangwen
>            Assignee: tangshangwen
>         Attachments: 0001-YARN-5795.patch
>
>
> In our cluster, we use the linux container, I would like to increase the number of cpu
to get more CPU time slice, but it did not take effect´╝î i set yarn.app.mapreduce.am.resource.cpu-vcores
= 2 ,but i found the resourcemanager log
> {noformat}
> [2016-10-27T16:36:37.280 08:00] [INFO] resourcemanager.scheduler.SchedulerNode.allocateContainer(SchedulerNode.java
153) [ResourceManager Event Processor] : Assigned container container_1477059529836_336635_01_000001
of capacity <memory:2048, vCores:1>
> {noformat}
> Because scheduler.getResourceCalculator() only computes memory
> {code:title=RMAppManager.java|borderStyle=solid}
>  private ResourceRequest validateAndCreateResourceRequest(
>       ApplicationSubmissionContext submissionContext, boolean isRecovery)
>       throws InvalidResourceRequestException {
> ...
>       SchedulerUtils.normalizeRequest(amReq, scheduler.getResourceCalculator(),
>           scheduler.getClusterResource(),
>           scheduler.getMinimumResourceCapability(),
>           scheduler.getMaximumResourceCapability(),
>           scheduler.getMinimumResourceCapability());
> ...
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message