hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Graves (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4045) Negative avaialbleMB is being reported for root queue.
Date Tue, 11 Aug 2015 17:19:47 GMT

    [ https://issues.apache.org/jira/browse/YARN-4045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14682115#comment-14682115
] 

Thomas Graves commented on YARN-4045:
-------------------------------------

I remember seeing that this was fixed in branch-2 by some of the capacity scheduler work for
labels.

I thought this might be fixed by https://issues.apache.org/jira/browse/YARN-3243 but that
is included.  

This might be fixed as part of https://issues.apache.org/jira/browse/YARN-3361 which is probably
to big to backport totally.

[~leftnoteasy]  Do you remember this issue?

Note that it also shows up in capacity scheduler UI as root queue going over 100%.  I remember
when I was testing YARN-3434 it wasn't occurring for me on branch-2 (2.8) and I thought it
was one of the above jiras that fixed.

> Negative avaialbleMB is being reported for root queue.
> ------------------------------------------------------
>
>                 Key: YARN-4045
>                 URL: https://issues.apache.org/jira/browse/YARN-4045
>             Project: Hadoop YARN
>          Issue Type: Bug
>    Affects Versions: 2.7.1
>            Reporter: Rushabh S Shah
>
> We recently deployed 2.7 in one of our cluster.
> We are seeing negative availableMB being reported for queue=root.
> This is from the jmx output:
> {noformat}
> <clusterMetrics>
>     ...
>     <availableMB>-163328</availableMB>
>     ...
> </clusterMetrics>
> {noformat}
> The following is the RM log:
> {noformat}
> 2015-08-10 14:42:28,280 [ResourceManager Event Processor] INFO capacity.ParentQueue:
completedContainer queue=root usedCapacity=1.0029854 absoluteUsedCapacity=1.0029854 used=<memory:5332480,
vCores:6202> cluster=<memory:5316608, vCores:28320>
> 2015-08-10 14:42:28,404 [ResourceManager Event Processor] INFO capacity.ParentQueue:
assignedContainer queue=root usedCapacity=1.0032743 absoluteUsedCapacity=1.0032743 used=<memory:5334016,
vCores:6212> cluster=<memory:5316608, vCores:28320>
> 2015-08-10 14:42:30,913 [ResourceManager Event Processor] INFO capacity.ParentQueue:
completedContainer queue=root usedCapacity=1.0029854 absoluteUsedCapacity=1.0029854 used=<memory:5332480,
vCores:6202> cluster=<memory:5316608, vCores:28320>
> 2015-08-10 14:42:30,913 [ResourceManager Event Processor] INFO capacity.ParentQueue:
assignedContainer queue=root usedCapacity=1.0032743 absoluteUsedCapacity=1.0032743 used=<memory:5334016,
vCores:6212> cluster=<memory:5316608, vCores:28320>
> 2015-08-10 14:42:33,093 [ResourceManager Event Processor] INFO capacity.ParentQueue:
completedContainer queue=root usedCapacity=1.0029854 absoluteUsedCapacity=1.0029854 used=<memory:5332480,
vCores:6202> cluster=<memory:5316608, vCores:28320>
> 2015-08-10 14:42:33,093 [ResourceManager Event Processor] INFO capacity.ParentQueue:
assignedContainer queue=root usedCapacity=1.0032743 absoluteUsedCapacity=1.0032743 used=<memory:5334016,
vCores:6212> cluster=<memory:5316608, vCores:28320>
> 2015-08-10 14:42:35,548 [ResourceManager Event Processor] INFO capacity.ParentQueue:
completedContainer queue=root usedCapacity=1.0029854 absoluteUsedCapacity=1.0029854 used=<memory:5332480,
vCores:6202> cluster=<memory:5316608, vCores:28320>
> 2015-08-10 14:42:35,549 [ResourceManager Event Processor] INFO capacity.ParentQueue:
assignedContainer queue=root usedCapacity=1.0032743 absoluteUsedCapacity=1.0032743 used=<memory:5334016,
vCores:6212> cluster=<memory:5316608, vCores:28320>
> 2015-08-10 14:42:39,088 [ResourceManager Event Processor] INFO capacity.ParentQueue:
completedContainer queue=root usedCapacity=1.0029854 absoluteUsedCapacity=1.0029854 used=<memory:5332480,
vCores:6202> cluster=<memory:5316608, vCores:28320>
> 2015-08-10 14:42:39,089 [ResourceManager Event Processor] INFO capacity.ParentQueue:
assignedContainer queue=root usedCapacity=1.0032743 absoluteUsedCapacity=1.0032743 used=<memory:5334016,
vCores:6212> cluster=<memory:5316608, vCores:28320>
> 2015-08-10 14:42:39,338 [ResourceManager Event Processor] INFO capacity.ParentQueue:
completedContainer queue=root usedCapacity=1.0029854 absoluteUsedCapacity=1.0029854 used=<memory:5332480,
vCores:6202> cluster=<memory:5316608, vCores:28320>
> 2015-08-10 14:42:39,339 [ResourceManager Event Processor] INFO capacity.ParentQueue:
assignedContainer queue=root usedCapacity=1.0032743 absoluteUsedCapacity=1.0032743 used=<memory:5334016,
vCores:6212> cluster=<memory:5316608, vCores:28320>
> 2015-08-10 14:42:39,757 [ResourceManager Event Processor] INFO capacity.ParentQueue:
completedContainer queue=root usedCapacity=1.0029854 absoluteUsedCapacity=1.0029854 used=<memory:5332480,
vCores:6202> cluster=<memory:5316608, vCores:28320>
> 2015-08-10 14:42:39,758 [ResourceManager Event Processor] INFO capacity.ParentQueue:
assignedContainer queue=root usedCapacity=1.0032743 absoluteUsedCapacity=1.0032743 used=<memory:5334016,
vCores:6212> cluster=<memory:5316608, vCores:28320>
> 2015-08-10 14:42:43,056 [ResourceManager Event Processor] INFO capacity.ParentQueue:
completedContainer queue=root usedCapacity=1.0029854 absoluteUsedCapacity=1.0029854 used=<memory:5332480,
vCores:6202> cluster=<memory:5316608, vCores:28320>
> 2015-08-10 14:42:43,070 [ResourceManager Event Processor] INFO capacity.ParentQueue:
assignedContainer queue=root usedCapacity=1.0032743 absoluteUsedCapacity=1.0032743 used=<memory:5334016,
vCores:6212> cluster=<memory:5316608, vCores:28320>
> 2015-08-10 14:42:44,486 [ResourceManager Event Processor] INFO capacity.ParentQueue:
completedContainer queue=root usedCapacity=1.0029854 absoluteUsedCapacity=1.0029854 used=<memory:5332480,
vCores:6202> cluster=<memory:5316608, vCores:28320>
> 2015-08-10 14:42:44,487 [ResourceManager Event Processor] INFO capacity.ParentQueue:
assignedContainer queue=root usedCapacity=1.0032743 absoluteUsedCapacity=1.0032743 used=<memory:5334016,
vCores:6212> cluster=<memory:5316608, vCores:28320>
> 2015-08-10 14:42:44,886 [ResourceManager Event Processor] INFO capacity.ParentQueue:
completedContainer queue=root usedCapacity=1.0029854 absoluteUsedCapacity=1.0029854 used=<memory:5332480,
vCores:6202> cluster=<memory:5316608, vCores:28320>
> 2015-08-10 14:42:44,886 [ResourceManager Event Processor] INFO capacity.ParentQueue:
assignedContainer queue=root usedCapacity=1.0032743 absoluteUsedCapacity=1.0032743 used=<memory:5334016,
vCores:6212> cluster=<memory:5316608, vCores:28320>
> 2015-08-10 14:42:47,401 [ResourceManager Event Processor] INFO capacity.ParentQueue:
completedContainer queue=root usedCapacity=1.0029854 absoluteUsedCapacity=1.0029854 used=<memory:5332480,
vCores:6202> cluster=<memory:5316608, vCores:28320>
> {noformat}
> bq.  used=<memory:5332480, vCores:6202> cluster=<memory:5316608, vCores:28320>
> For root queue, usedCapacity is more than totalCapacity



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message