Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 11FC810965 for ; Fri, 5 Dec 2014 19:22:28 +0000 (UTC) Received: (qmail 73495 invoked by uid 500); 5 Dec 2014 19:22:23 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 73368 invoked by uid 500); 5 Dec 2014 19:22:23 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 73358 invoked by uid 99); 5 Dec 2014 19:22:23 -0000 Received: from mail-relay.apache.org (HELO mail-relay.apache.org) (140.211.11.15) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 05 Dec 2014 19:22:23 +0000 Received: from spacestar.att.net (107-222-184-213.lightspeed.sntcca.sbcglobal.net [107.222.184.213]) by mail-relay.apache.org (ASF Mail Server at mail-relay.apache.org) with ESMTPSA id 964351A0019 for ; Fri, 5 Dec 2014 19:22:21 +0000 (UTC) From: Vinod Kumar Vavilapalli Content-Type: multipart/signed; boundary="Apple-Mail=_CCB78185-51D2-4B98-8E56-C1510C3F3E3F"; protocol="application/pgp-signature"; micalg=pgp-sha1 Message-Id: <69DEBEF3-5833-4BE5-86C2-7FF2A74E2120@apache.org> Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: When schedulers consider x% of resources what do they mean? Date: Fri, 5 Dec 2014 11:22:19 -0800 References: To: user@hadoop.apache.org In-Reply-To: X-Mailer: Apple Mail (2.1878.6) --Apple-Mail=_CCB78185-51D2-4B98-8E56-C1510C3F3E3F Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Resources can mean memory-only (by default) or memory + CPU etc across = the _entire_ cluster. So 70% of cluster resources for a queue means that 70% of the total = memory set for Hadoop in the cluster are available for all applications = in that queue. Heap sizes are part of the memory requirements for each container. HTH +Vinod On Dec 5, 2014, at 5:41 AM, Chris Mawata wrote: > Hi all, > when you divide up resources e.g. on CapacityScheduler or = FairScheduler etc., what does x% or resources mean? So, for example, a = guranteed 70% meant to indicate you can have up to > 70% of the containers clusterwide irrespective of size of container=20 > 70% of the containers on each node=20 > or should it not be number of containers but sum of heap sizes? >=20 > Cheers > Chris Mawata --Apple-Mail=_CCB78185-51D2-4B98-8E56-C1510C3F3E3F Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJUggXrAAoJECVa31bDbF8P3XEH/jIk9ugNSLbHZLB5WLxrdk2k QuuzWWLkRmL14R2mX1k6TrolPHqD1ihp0UttrWYz02rw6Bd5ZAXeOvpLYK3XUz7A P7FtuyoWCLQ/dHSX3j7VVHrgqsR4Fme99d5lSwnQyruPMGd4+5Oit88vVTMeFdd5 14IPdxjNfGASSMXASRUZUH8S0znBDgVenjUxMnEaMnyK7Nv/mlpEUGzM60glIS95 Mt9LqULrlwCJRBzf+l/Pga3Y880/6Zp/UrR3EBbj6MESbQjmko2Y3d7enfn0wr73 K3x75oLxGzgnakl7+gVd1L/qvZQPfS5S761eQZc782wcc2BqKgAKTgPIZFUsvBw= =/dcA -----END PGP SIGNATURE----- --Apple-Mail=_CCB78185-51D2-4B98-8E56-C1510C3F3E3F--