hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sunil G (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4678) Cluster used capacity is > 100 when container reserved
Date Fri, 11 Mar 2016 16:57:39 GMT

    [ https://issues.apache.org/jira/browse/YARN-4678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15191208#comment-15191208

Sunil G commented on YARN-4678:

[~brahmareddy] Thanks for sharing the scenario.

We have made changes only to parent queue in current patch. We could have been done similar
change in LeafQueue, but I had one concern. To select an under serving queue while processing
a node heartbeat, we still consider usedCapacity(inclusive of reservedCapacity), so there
might be difference is showing queue capacity on these two cases. I still feel its fine to
go ahead with UI changes as reservation metrics are displayed in LeafQueue page.. Let me make
change to consider this scenario. Meantime if you or [~bibinchundatt] see the above mentioned
scenario as a pblm, pls share your thoughts.

> Cluster used capacity is > 100 when container reserved 
> -------------------------------------------------------
>                 Key: YARN-4678
>                 URL: https://issues.apache.org/jira/browse/YARN-4678
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Brahma Reddy Battula
>            Assignee: Sunil G
>         Attachments: 0001-YARN-4678.patch, 0002-YARN-4678.patch
>  *Scenario:* 
> * Start cluster with Three NM's each having 8GB (cluster memory:24GB).
> * Configure queues with elasticity and userlimitfactor=10.
> * disable pre-emption.
> * run two job with different priority in different queue at the same time
> ** yarn jar hadoop-mapreduce-examples-2.7.2.jar pi -Dyarn.app.priority=LOW -Dmapreduce.job.queuename=QueueA
-Dmapreduce.map.memory.mb=4096 -Dyarn.app.mapreduce.am.resource.mb=1536 -Dmapreduce.job.reduce.slowstart.completedmaps=1.0
10 1000000000000
> ** yarn jar hadoop-mapreduce-examples-2.7.2.jar pi -Dyarn.app.priority=HIGH -Dmapreduce.job.queuename=QueueB
-Dmapreduce.map.memory.mb=4096 -Dyarn.app.mapreduce.am.resource.mb=1536 3 1000000000000
> * observe the cluster capacity which was used in RM web UI

This message was sent by Atlassian JIRA

View raw message