hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4538) QueueMetrics pending cores and memory metrics wrong
Date Wed, 06 Jan 2016 18:43:39 GMT

    [ https://issues.apache.org/jira/browse/YARN-4538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15086037#comment-15086037
] 

Wangda Tan commented on YARN-4538:
----------------------------------

Hi [~bibinchundatt],
Thanks for working on the patch, looked at your patch, I think this patch may break some existing
scenario, let me explain:

For existing workflow:
- Container increase/decrease: call incr/decrPendingResource of QueueMetrics, Parameter is:
#container=0, resource=delta-resource. And incr/decrPendingResource should not change #container,
but should update pendingResource by resource
- Container allocation/release:  call incr/decrPendingResource of QueueMetrics, Parameter
is: #container should > 0, resource=per-container-resource, pendingResource should be updated
by #container * resource

So under this assumption, when container increase/decrease, your existing patch will skip
updating pendingResource.

Suggestion of fixes:
- Choice#1, change all existing caller of incr/decrPendingResource for container allocation/release
to make sure that they will be called when #container>0
- Choice#2, create a new method for increase/decrease container to change resource but not
update #container in QueueMetrics.

I would prefer choice#2. 

Thoughts?

> QueueMetrics pending  cores and memory metrics wrong
> ----------------------------------------------------
>
>                 Key: YARN-4538
>                 URL: https://issues.apache.org/jira/browse/YARN-4538
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Bibin A Chundatt
>            Assignee: Bibin A Chundatt
>         Attachments: 0001-YARN-4538.patch
>
>
> Submit 2 application to default queue 
> Check queue metrics for pending cores and memory
> {noformat}
>     List<QueueInfo> allQueues = client.getChildQueueInfos("root");
>     for (QueueInfo queueInfo : allQueues) {
>       QueueStatistics quastats = queueInfo.getQueueStatistics();
>       System.out.println(quastats.getPendingVCores());
>       System.out.println(quastats.getPendingMemoryMB());
>     }
> {noformat}
> *Output :*
> -20
> -20480



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message