hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4678) Cluster used capacity is > 100 when container reserved
Date Wed, 23 Mar 2016 22:25:25 GMT

    [ https://issues.apache.org/jira/browse/YARN-4678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15209298#comment-15209298
] 

Wangda Tan commented on YARN-4678:
----------------------------------

Hi [~sunilg],

Thanks for working on this JIRA, it is useful to record reserved resources separately.

However, I'm thinking how this could happen: ParentQueue's capacity will be checked when we
reserve container, we should make sure that allocation of reserved container shouldn't violate
parent queue's max capacity.

> Cluster used capacity is > 100 when container reserved 
> -------------------------------------------------------
>
>                 Key: YARN-4678
>                 URL: https://issues.apache.org/jira/browse/YARN-4678
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Brahma Reddy Battula
>            Assignee: Sunil G
>         Attachments: 0001-YARN-4678.patch, 0002-YARN-4678.patch, 0003-YARN-4678.patch
>
>
>  *Scenario:* 
> * Start cluster with Three NM's each having 8GB (cluster memory:24GB).
> * Configure queues with elasticity and userlimitfactor=10.
> * disable pre-emption.
> * run two job with different priority in different queue at the same time
> ** yarn jar hadoop-mapreduce-examples-2.7.2.jar pi -Dyarn.app.priority=LOW -Dmapreduce.job.queuename=QueueA
-Dmapreduce.map.memory.mb=4096 -Dyarn.app.mapreduce.am.resource.mb=1536 -Dmapreduce.job.reduce.slowstart.completedmaps=1.0
10 1000000000000
> ** yarn jar hadoop-mapreduce-examples-2.7.2.jar pi -Dyarn.app.priority=HIGH -Dmapreduce.job.queuename=QueueB
-Dmapreduce.map.memory.mb=4096 -Dyarn.app.mapreduce.am.resource.mb=1536 3 1000000000000
> * observe the cluster capacity which was used in RM web UI



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message