hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Filipp Zhinkin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6378) Negative usedResources memory in CapacityScheduler
Date Tue, 04 Jul 2017 07:54:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-6378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16073266#comment-16073266
] 

Filipp Zhinkin commented on YARN-6378:
--------------------------------------

Seems like hadoop 2.6.0 has the same problem, at least I'm facing negative memory and vcores
usage with capacity scheduler:

{code:xml}
<queue xsi:type="capacitySchedulerLeafQueueInfo">
        <capacity>19.0</capacity>
        <usedCapacity>-69.52686</usedCapacity>
        <maxCapacity>90.0</maxCapacity>
        <absoluteCapacity>19.0</absoluteCapacity>
        <absoluteMaxCapacity>90.0</absoluteMaxCapacity>
        <absoluteUsedCapacity>0.0</absoluteUsedCapacity>
        <numApplications>10</numApplications>
        <queueName>default</queueName>
        <state>RUNNING</state>
        <resourcesUsed>
           <memory>-152576</memory>
           <vCores>-41</vCores>
        </resourcesUsed>
        <hideReservationQueues>false</hideReservationQueues>
        <nodeLabels>*</nodeLabels>
        <allocatedContainers>24</allocatedContainers>
        <reservedContainers>0</reservedContainers>
        <pendingContainers>0</pendingContainers>
        <numActiveApplications>10</numActiveApplications>
        <numPendingApplications>0</numPendingApplications>
        <numContainers>-41</numContainers>
        <maxApplications>1900</maxApplications>
        <maxApplicationsPerUser>855</maxApplicationsPerUser>
        <maxActiveApplications>102</maxActiveApplications>
        <maxActiveApplicationsPerUser>10</maxActiveApplicationsPerUser>
        <userLimit>10</userLimit>
        ...
  </queue>
{code}


> Negative usedResources memory in CapacityScheduler
> --------------------------------------------------
>
>                 Key: YARN-6378
>                 URL: https://issues.apache.org/jira/browse/YARN-6378
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacity scheduler, resourcemanager
>    Affects Versions: 2.7.2
>            Reporter: Ravi Prakash
>            Assignee: Ravi Prakash
>
> Courtesy Thomas Nystrand, we found that on two of our clusters configured with the CapacityScheduler,
usedResources occasionally becomes negative. 
> e.g.
> {code}
> 2017-03-15 11:10:09,449 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue:
assignedContainer application attempt=appattempt_1487222361993_17177_000001 container=Container:
[ContainerId: container_1487222361993_17177_01_000014, NodeId: <SOMENODE>:27249, NodeHttpAddress:
<SOMENODE>:8042, Resource: <memory:6656, vCores:1>, Priority: 2, Token: null,
] queue=<somequeuename>: capacity=0.2, absoluteCapacity=0.2, usedResources=<memory:-1024,
vCores:3>, usedCapacity=0.03409091, absoluteUsedCapacity=0.006818182, numApps=1, numContainers=3
clusterResource=<memory:1249280, vCores:440> type=RACK_LOCAL
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message