Return-Path: X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id C9DC8104AB for ; Sat, 1 Jun 2013 00:01:00 +0000 (UTC) Received: (qmail 26755 invoked by uid 500); 1 Jun 2013 00:00:56 -0000 Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org Received: (qmail 26414 invoked by uid 500); 1 Jun 2013 00:00:55 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 26406 invoked by uid 99); 1 Jun 2013 00:00:55 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 01 Jun 2013 00:00:55 +0000 X-ASF-Spam-Status: No, hits=0.7 required=5.0 tests=RCVD_IN_DNSWL_NONE,SPF_PASS,UNRESOLVED_TEMPLATE X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of chuanliu@microsoft.com designates 207.46.163.151 as permitted sender) Received: from [207.46.163.151] (HELO na01-bn1-obe.outbound.protection.outlook.com) (207.46.163.151) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 01 Jun 2013 00:00:46 +0000 Received: from BL2FFO11FD016.protection.gbl (10.173.161.203) by BL2FFO11HUB002.protection.gbl (10.173.161.20) with Microsoft SMTP Server (TLS) id 15.0.707.0; Sat, 1 Jun 2013 00:00:24 +0000 Received: from TK5EX14MLTC104.redmond.corp.microsoft.com (131.107.125.37) by BL2FFO11FD016.mail.protection.outlook.com (10.173.160.224) with Microsoft SMTP Server (TLS) id 15.0.707.0 via Frontend Transport; Sat, 1 Jun 2013 00:00:24 +0000 Received: from ch1outboundpool.messaging.microsoft.com (157.54.51.112) by mail.microsoft.com (157.54.79.159) with Microsoft SMTP Server (TLS) id 14.2.318.3; Fri, 31 May 2013 23:59:50 +0000 Received: from mail43-ch1-R.bigfish.com (10.43.68.231) by CH1EHSOBE019.bigfish.com (10.43.70.76) with Microsoft SMTP Server id 14.1.225.23; Fri, 31 May 2013 23:59:27 +0000 Received: from mail43-ch1 (localhost [127.0.0.1]) by mail43-ch1-R.bigfish.com (Postfix) with ESMTP id 6A82DC040F for ; Fri, 31 May 2013 23:59:27 +0000 (UTC) X-Forefront-Antispam-Report-Untrusted: CIP:157.56.240.21;KIP:(null);UIP:(null);(null);H:BL2PRD0310HT002.namprd03.prod.outlook.com;R:internal;EFV:INT X-SpamScore: 1 X-BigFish: PS1(zz9371I542I4015Izz1f42h1ee6h1de0h1fdah1202h1e76h1d1ah1d2ah1fc6hz8dhz8275dhz31h2a8h668h839h944hd24hf0ah1220h1288h12a5h12a9h12bdh137ah13b6h1441h1504h1537h153bh162dh1631h1758h18e1h1946h19b5h1ad9h1b0ah1d07h1d0ch1d2eh1d3fh1de9h1dfeh1dffh17ej9a9j1155h) Received-SPF: softfail (mail43-ch1: transitioning domain of microsoft.com does not designate 157.56.240.21 as permitted sender) client-ip=157.56.240.21; envelope-from=chuanliu@microsoft.com; helo=BL2PRD0310HT002.namprd03.prod.outlook.com ;.outlook.com ; X-Forefront-Antispam-Report-Untrusted: SFV:SKI;SFS:;DIR:OUT;SFP:;SCL:-1;SRVR:BY2PR03MB621;H:BY2PR03MB621.namprd03.prod.outlook.com;LANG:en; Received: from mail43-ch1 (localhost.localdomain [127.0.0.1]) by mail43-ch1 (MessageSwitch) id 1370044766136984_5296; Fri, 31 May 2013 23:59:26 +0000 (UTC) Received: from CH1EHSMHS043.bigfish.com (snatpool2.int.messaging.microsoft.com [10.43.68.231]) by mail43-ch1.bigfish.com (Postfix) with ESMTP id 15761480049; Fri, 31 May 2013 23:59:26 +0000 (UTC) Received: from BL2PRD0310HT002.namprd03.prod.outlook.com (157.56.240.21) by CH1EHSMHS043.bigfish.com (10.43.69.252) with Microsoft SMTP Server (TLS) id 14.1.225.23; Fri, 31 May 2013 23:59:25 +0000 Received: from BY2PR03MB621.namprd03.prod.outlook.com (10.255.93.43) by BL2PRD0310HT002.namprd03.prod.outlook.com (10.255.97.37) with Microsoft SMTP Server (TLS) id 14.16.311.1; Fri, 31 May 2013 23:59:25 +0000 Received: from BY2PR03MB621.namprd03.prod.outlook.com (10.255.93.43) by BY2PR03MB621.namprd03.prod.outlook.com (10.255.93.43) with Microsoft SMTP Server (TLS) id 15.0.698.13; Fri, 31 May 2013 23:59:22 +0000 Received: from BY2PR03MB621.namprd03.prod.outlook.com ([169.254.7.22]) by BY2PR03MB621.namprd03.prod.outlook.com ([169.254.7.22]) with mapi id 15.00.0698.010; Fri, 31 May 2013 23:59:22 +0000 From: Chuan Liu To: "yuzhanghan1982@gmail.com" CC: "user@hadoop.apache.org" Subject: RE: How to configure container capacity? Thread-Topic: How to configure container capacity? Thread-Index: AQHOXjaK2AgVJtWfakKVNM5g7opyhpkf9ktQ Date: Fri, 31 May 2013 23:59:21 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [2001:4898:23:4:b870:c59a:1530:1dc6] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OrganizationHeadersPreserved: BY2PR03MB621.namprd03.prod.outlook.com X-FOPE-CONNECTOR: Id%0$Dn%*$RO%0$TLS%0$FQDN%$TlsDn% X-FOPE-CONNECTOR: Id%59$Dn%HADOOP.APACHE.ORG$RO%2$TLS%6$FQDN%corpf5vips-237160.customer.frontbridge.com$TlsDn% X-FOPE-CONNECTOR: Id%59$Dn%GMAIL.COM$RO%2$TLS%6$FQDN%corpf5vips-237160.customer.frontbridge.com$TlsDn% X-CrossPremisesHeadersPromoted: TK5EX14MLTC104.redmond.corp.microsoft.com X-CrossPremisesHeadersFiltered: TK5EX14MLTC104.redmond.corp.microsoft.com X-Forefront-Antispam-Report: CIP:131.107.125.37;CTRY:US;IPV:CAL;IPV:NLI;EFV:NLI;SFV:NSPM;SFS:(13464003)(189002)(164054003)(377454002)(199002)(53806001)(16676001)(6806003)(47976001)(47446002)(49866001)(76796001)(74502001)(1411001)(51856001)(77982001)(76482001)(50466002)(79102001)(81342001)(31966008)(4396001)(76786001)(74316001)(54356001)(50986001)(54316002)(76576001)(56816002)(59766001)(56776001)(46102001)(46406003)(47776003)(20776003)(33646001)(47736001)(74366001)(44976003)(74876001)(74662001)(69226001)(74706001)(23726002)(81542001)(65816001)(80022001)(63696002)(24736002)(3826001);DIR:OUT;SFP:;SCL:1;SRVR:BL2FFO11HUB002;H:TK5EX14MLTC104.redmond.corp.microsoft.com;CLIP:131.107.125.37;RD:InfoDomainNonexistent;MX:1;A:1;LANG:en; X-OriginatorOrg: microsoft.onmicrosoft.com X-Forefront-PRVS: 0864A36BBF X-Virus-Checked: Checked by ClamAV on apache.org Bcc'd dev mailing list. Hi Andrew, The memory allocated will always be an integral multiple of minimal allocat= ion unit which is configured via the property "yarn.scheduler.minimum-alloc= ation-mb ". The default configuration for the value is 1024. If you change the config to 512, the container memory size should be 512 wi= th -Dmapreduce.map.memory.mb=3D512. Thanks, Chuan -----Original Message----- From: Yuzhang Han [mailto:yuzhanghan1982@gmail.com]=20 Sent: Friday, May 31, 2013 12:24 PM To: yarn-dev-subscribe@hadoop.apache.org; yarn-dev@hadoop.apache.org Subject: How to configure container capacity? Hi, Can anyone tell me how to configure the capacity of containers? Which param= eters should be set? I use the following command with -Dmapreduce.map.memory.mb=3D512, but why i= s the map container size always 1024? $HADOOP_COMMON_HOME/bin/hadoop jar $HADOOP_MAPRED_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.0.4-= alpha.jar randomwriter -Dmapreduce.job.user.name=3D$USER -Dmapreduce.randomwriter.bytespermap=3D10000 -Ddfs.block.size=3D536870912 -Dmapreduce.map.memory.mb=3D512 output Thank you, Andrew