Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 36198DFCA for ; Sat, 9 Feb 2013 13:07:17 +0000 (UTC) Received: (qmail 40928 invoked by uid 500); 9 Feb 2013 13:07:12 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 40188 invoked by uid 500); 9 Feb 2013 13:07:06 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Delivered-To: moderator for user@hadoop.apache.org Received: (qmail 36317 invoked by uid 99); 9 Feb 2013 13:05:11 -0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of rajeshbabu.chintaguntla@huawei.com designates 119.145.14.65 as permitted sender) From: rajeshbabu chintaguntla To: "acm@hortonworks.com" CC: "mapreduce-dev@hadoop.apache.org" , "user@hadoop.apache.org" Subject: maximum capacity of queue is not effecting beyond its capacity Thread-Topic: maximum capacity of queue is not effecting beyond its capacity Thread-Index: Ac4GunbKwtnTcNcJQYWhdfAXV5xzCQ== Date: Sat, 9 Feb 2013 13:04:35 +0000 Message-ID: Accept-Language: en-US, zh-CN Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.18.96.96] Content-Type: multipart/alternative; boundary="_000_B2048EB0A868B54F9FB50119F14D829702D47BF7szxeml533mbschi_" MIME-Version: 1.0 X-CFilter-Loop: Reflected X-Virus-Checked: Checked by ClamAV on apache.org --_000_B2048EB0A868B54F9FB50119F14D829702D47BF7szxeml533mbschi_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi, When I run a job its getting hanged bacause its not able to get free memory= resources to map/reduce task containers. Total Memory avialable : 8 GB schedular configured : CapacitySchedular queue path cofigured : a Application master got started on 'a' queue path by consuming 1.5 GB Afterthat no container is getting launched because its not getting free me= mory from 'a' queue even free memory(max - cosumed) is sufficient. When I debugged I observed queue capacity requirements are satified but use= r capacity requirements are not satified. There is some problem I think. I am not able to follow the logic behind it.= Please help me in this case. capacity-schedular.xml - configurations =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D yarn.scheduler.capacity.root.queues a,b The queues at the this level (root is the root queue). yarn.scheduler.capacity.root.capacity 100 yarn.scheduler.capacity.root.a.capacity 5 yarn.scheduler.capacity.root.a.maximum-capacity 60 yarn.scheduler.capacity.root.b.capacity 95 yarn.scheduler.capacity.maximum-am-resource-percent 8 yarn.scheduler.capacity.root.acl_submit_jobs * yarn.scheduler.capacity.root.a.acl_submit_jobs * yarn.scheduler.capacity.root.b.acl_submit_jobs * Thanks and Regards, Rajeshbabu. --_000_B2048EB0A868B54F9FB50119F14D829702D47BF7szxeml533mbschi_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Hi,

 

When I run a job its getting hanged bacause its not able to get free mem= ory resources to map/reduce task containers.

 

Total Memory avialable : 8 GB

schedular configured : CapacitySchedular

queue path cofigured : a

 

Application master got started on 'a' queue path by consu= ming 1.5 GB

 

Afterthat  no container is getting launched because its not getting= free memory from 'a' queue even free memory(max - cosumed) is suffici= ent.

When I debugged I observed queue capacity requirements are satified but = user capacity requirements are not satified.

 

There is some problem I think. I am not able to follow the logic behind = it. Please help me in this case.

 

capacity-schedular.xml - configurations

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=

<configurations>

<property>
  <name>yarn.scheduler.capacity.root.queues</name>
  <value>a,b</value>
  <description>The queues at the this level (root is the root qu= eue).
  </description>
</property>

<property>

<name>yarn.s= cheduler.capacity.root.capacity</name>

<value>100<value>

</property>

<property>

<name>yarn.s= cheduler.capacity.root.a.capacity</name>

<value>5<value>

</property>

<property>

<name>yarn.s= cheduler.capacity.root.a.maximum-capacity</name>

<value>60<value>

</property>

<property>

<name>yarn.s= cheduler.capacity.root.b.capacity</name>

<value>95<value>

</property>

<property>

<name>yarn.s= cheduler.capacity.maximum-am-resource-percent</name>

<value>8<value>

</property>

<property>

<name>yarn.scheduler.capacity.root.acl_= submit_jobs</name>

<value>*<value>

</property>
<property>

<name>yarn.scheduler.capacity.root.a.ac= l_submit_jobs</name>

<value>*<value>

</property>

<property>

<name>yarn.scheduler.capacity.root.b.ac= l_submit_jobs</name>

<value>*<value>

</property>

</configurations>

 

Thanks and Regards,

Rajeshbabu.

--_000_B2048EB0A868B54F9FB50119F14D829702D47BF7szxeml533mbschi_--