hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "tangshangwen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5795) FairScheduler set AppMaster vcores didn't work
Date Fri, 28 Oct 2016 08:49:58 GMT

    [ https://issues.apache.org/jira/browse/YARN-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15614804#comment-15614804
] 

tangshangwen commented on YARN-5795:
------------------------------------

hi [~kasha], i think the DefaultResourceCalculator in the normalize method should be to maintain
the original vcore,
{code:title=DefaultResourceCalculator.java|borderStyle=solid}
  @Override
  public Resource normalize(Resource r, Resource minimumResource,
      Resource maximumResource, Resource stepFactor) {
    int normalizedMemory = Math.min(
        roundUp(
            Math.max(r.getMemory(), minimumResource.getMemory()),
            stepFactor.getMemory()),
            maximumResource.getMemory());
    return Resources.createResource(normalizedMemory);
  }
{code}
change to
{code:title=DefaultResourceCalculator.java|borderStyle=solid}
  @Override
  public Resource normalize(Resource r, Resource minimumResource,
      Resource maximumResource, Resource stepFactor) {
    int normalizedMemory = Math.min(
        roundUp(
            Math.max(r.getMemory(), minimumResource.getMemory()),
            stepFactor.getMemory()),
            maximumResource.getMemory());
    return Resources.createResource(normalizedMemory, r.getVirtualCores());
  }
{code}

> FairScheduler set AppMaster vcores didn't work
> ----------------------------------------------
>
>                 Key: YARN-5795
>                 URL: https://issues.apache.org/jira/browse/YARN-5795
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: resourcemanager
>    Affects Versions: 2.7.1
>            Reporter: tangshangwen
>            Assignee: tangshangwen
>
> In our cluster, we use the linux container, I would like to increase the number of cpu
to get more CPU time slice, but it did not take effect´╝î i set yarn.app.mapreduce.am.resource.cpu-vcores
= 2 ,but i found the resourcemanager log
> {noformat}
> [2016-10-27T16:36:37.280 08:00] [INFO] resourcemanager.scheduler.SchedulerNode.allocateContainer(SchedulerNode.java
153) [ResourceManager Event Processor] : Assigned container container_1477059529836_336635_01_000001
of capacity <memory:2048, vCores:1>
> {noformat}
> Because scheduler.getResourceCalculator() only computes memory
> {code:title=RMAppManager.java|borderStyle=solid}
>  private ResourceRequest validateAndCreateResourceRequest(
>       ApplicationSubmissionContext submissionContext, boolean isRecovery)
>       throws InvalidResourceRequestException {
> ...
>       SchedulerUtils.normalizeRequest(amReq, scheduler.getResourceCalculator(),
>           scheduler.getClusterResource(),
>           scheduler.getMinimumResourceCapability(),
>           scheduler.getMaximumResourceCapability(),
>           scheduler.getMinimumResourceCapability());
> ...
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message