Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7DCE610E00 for ; Thu, 5 Dec 2013 01:23:31 +0000 (UTC) Received: (qmail 2916 invoked by uid 500); 5 Dec 2013 01:23:26 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 2817 invoked by uid 500); 5 Dec 2013 01:23:26 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 2810 invoked by uid 99); 5 Dec 2013 01:23:26 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 05 Dec 2013 01:23:26 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of justlooks@gmail.com designates 209.85.216.54 as permitted sender) Received: from [209.85.216.54] (HELO mail-qa0-f54.google.com) (209.85.216.54) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 05 Dec 2013 01:23:22 +0000 Received: by mail-qa0-f54.google.com with SMTP id f11so7274605qae.13 for ; Wed, 04 Dec 2013 17:23:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=tePso+ig8QpWZtbi5NrWNoBR3Z3KSbLoCsq7T3Ae+ew=; b=de/iT6Z/kTjYd/1yNimfvs+Ylzn751qIN1uMg+teXxcRGM9jU8+Po7P7cH6aQ1Xp5/ ARIvWHPTXsS921QQJ6LaLl2IQ/mMIvUL1ruoDnaY6uqOu0qTILwg6Z8ox5dln0Iv1ybp FbkghKAUeiLtLM8P3QUyGILCfk/zxDeupOwSyPT3F2XA7QO9rZTMoVzQbtZvFVjxqddw gRSKXg8dEMHE00Wq1ZvbR+exTR07mYqK4OOvPiplVWRfJCBv5jmgItCNXOLYvzbx0Ndg Z8uxaP6jxezwIQE4L7/KuTHGjUoJqYVQFgTqqD5WCBvNmkLQQIG3c+8PtO9aoSnpjUQL y+Zg== MIME-Version: 1.0 X-Received: by 10.224.51.196 with SMTP id e4mr123312471qag.16.1386206581444; Wed, 04 Dec 2013 17:23:01 -0800 (PST) Received: by 10.140.20.37 with HTTP; Wed, 4 Dec 2013 17:23:01 -0800 (PST) In-Reply-To: References: Date: Thu, 5 Dec 2013 09:23:01 +0800 Message-ID: Subject: Re: issue about capacity scheduler From: ch huang To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=001a11c29e50df834a04ecbf5e3c X-Virus-Checked: Checked by ClamAV on apache.org --001a11c29e50df834a04ecbf5e3c Content-Type: text/plain; charset=ISO-8859-1 another question is ,i set the yarn.scheduler.minimum-allocation-mb is 2GB,so the container size will at less 2GB ,but i see appMaster container only use 1GB heap size why? # ps -ef|grep 8062 yarn 8062 8047 5 09:04 ? 00:00:09 /usr/java/jdk1.7.0_25/bin/java -Dlog4j.configuration=container-log4j.properties -Dyarn.app.mapreduce.container.log.dir=/data/mrlocal/1/yarn/logs/application_1386139114497_0024/container_1386139114497_0024_01_000001 -Dyarn.app.mapreduce.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Xmx1024m org.apache.hadoop.mapreduce.v2.app.MRAppMaster On Thu, Dec 5, 2013 at 5:04 AM, Jian He wrote: > you can probably try increasing > "yarn.scheduler.capacity.maximum-am-resource-percent", > This controls the max concurrently running AMs. > > Thanks, > Jian > > > On Wed, Dec 4, 2013 at 1:33 AM, ch huang wrote: > >> hi,maillist : >> i use yarn framework and capacity scheduler ,and i have >> two queue ,one for hive and the other for big MR job >> in hive queue it's work fine,because hive task is very faster ,but what >> i think is user A submitted two big MR job ,so first big job eat >> all the resource belongs to the queue ,the other big MR job should wait >> until first job finished ,how can i let the same user 's MR job can run >> parallel? >> > > > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity > to which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. --001a11c29e50df834a04ecbf5e3c Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
another question is ,i set the yarn.scheduler.minimum-allocation-mb is= 2GB,so the container size will at less 2GB ,but i see appMaster container = only use 1GB heap size why?
=A0
# ps -ef|grep 8062
yarn=A0=A0=A0=A0=A0 8062=A0 8047=A0 5 09:04 ?=A0= =A0=A0=A0=A0=A0=A0 00:00:09 /usr/java/jdk1.7.0_25/bin/java -Dlog4j.configur= ation=3Dcontainer-log4j.properties -Dyarn.app.mapreduce.container.log.dir= =3D/data/mrlocal/1/yarn/logs/application_1386139114497_0024/container_13861= 39114497_0024_01_000001 -Dyarn.app.mapreduce.container.log.filesize=3D0 -Dh= adoop.root.logger=3DINFO,CLA -Xmx1024m org.apache.hadoop.mapreduce.v2.app.M= RAppMaster


On Thu, Dec 5, 2013 at 5:04 AM, Jian He <jhe@= hortonworks.com> wrote:
you can probably try increasing "yarn.scheduler.capac= ity.maximum-am-resource-percent",
This controls the max concurrentl= y running AMs.

Thanks,
Jian


On Wed, Dec 4, 2013 at 1:33 AM, ch huang <jus= tlooks@gmail.com> wrote:
hi,maillist :
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 i use yarn framework = and capacity scheduler=A0 ,and i have two queue ,one for hive and the other= for big MR job
in hive=A0 queue it's work fine,because hive task is very faster ,= but what i think is user A submitted two big MR job ,so first big job eat
all the resource belongs to the queue ,the other big MR job should wai= t until first job finished ,how can i let the same user 's MR job can r= un parallel?


CON= FIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or e= ntity to which it is addressed and may contain information that is confiden= tial, privileged and exempt from disclosure under applicable law. If the re= ader of this message is not the intended recipient, you are hereby notified= that any printing, copying, dissemination, distribution, disclosure or for= warding of this communication is strictly prohibited. If you have received = this communication in error, please contact the sender immediately and dele= te it from your system. Thank You.

--001a11c29e50df834a04ecbf5e3c--