Return-Path: X-Original-To: apmail-hive-user-archive@www.apache.org Delivered-To: apmail-hive-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id F3E5F18C0E for ; Tue, 13 Oct 2015 20:20:31 +0000 (UTC) Received: (qmail 15855 invoked by uid 500); 13 Oct 2015 20:20:30 -0000 Delivered-To: apmail-hive-user-archive@hive.apache.org Received: (qmail 15778 invoked by uid 500); 13 Oct 2015 20:20:30 -0000 Mailing-List: contact user-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hive.apache.org Delivered-To: mailing list user@hive.apache.org Received: (qmail 15768 invoked by uid 99); 13 Oct 2015 20:20:30 -0000 Received: from Unknown (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 13 Oct 2015 20:20:30 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 191A71A20DC for ; Tue, 13 Oct 2015 20:20:30 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 4.1 X-Spam-Level: **** X-Spam-Status: No, score=4.1 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=3, KAM_LINEPADDING=1.2, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-us-west.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id r3msvV_dls0I for ; Tue, 13 Oct 2015 20:20:19 +0000 (UTC) Received: from mail-ig0-f175.google.com (mail-ig0-f175.google.com [209.85.213.175]) by mx1-us-west.apache.org (ASF Mail Server at mx1-us-west.apache.org) with ESMTPS id BCCE920562 for ; Tue, 13 Oct 2015 20:20:18 +0000 (UTC) Received: by igbkq10 with SMTP id kq10so100789135igb.0 for ; Tue, 13 Oct 2015 13:20:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=qgBiKG5TnJhVgr6fndO9yKUmfZSk8D85JYf3gjynQcY=; b=KHo+Uw6mdvmYPApfYES3VE9Rc1ifamtYBU8M5ZfYQ5Z1VO04QJ7CXPfYPuMnnAOGAr hHS3k4+l8lBqCVxqw2slP48gjdyDeHhMYMnITTa2mN9NSa0gD3B0PQTzgZGcGKpewrrj vmFPcTclWCK/f2xpTPXlrW5+9gmVU5bcGfSifsKS/hYa+kXUxLIOQzlTPsFe7DbZD8X5 vYP1EmBh+xXcgKjBVTBmE7q2Ze7MHUZa+F9SQ4cFSLI0wJRhzTWGS3daL2QbIWsK8gZG m69uBwLakiFpue80MjlfZFDNuDq0HyrUqKMQAMWWOa/imikaoEZ3CMTNUHFLgCIY+CUm +TOQ== MIME-Version: 1.0 X-Received: by 10.50.43.170 with SMTP id x10mr21931142igl.89.1444767618171; Tue, 13 Oct 2015 13:20:18 -0700 (PDT) Received: by 10.36.73.196 with HTTP; Tue, 13 Oct 2015 13:20:18 -0700 (PDT) In-Reply-To: <00a801d105f3$79000010$6b000030$@peridale.co.uk> References: <009101d105d4$9446ab40$bcd401c0$@peridale.co.uk> <00a801d105f3$79000010$6b000030$@peridale.co.uk> Date: Wed, 14 Oct 2015 01:50:18 +0530 Message-ID: Subject: Re: Container is running beyond physical memory limits From: hadoop hive To: user@hive.apache.org Content-Type: multipart/alternative; boundary=089e01176cadaa26e10522022c27 --089e01176cadaa26e10522022c27 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable http://hortonworks.com/blog/how-to-plan-and-configure-yarn-in-hdp-2-0/ On Wed, Oct 14, 2015 at 1:42 AM, Mich Talebzadeh wrote: > Thank you all. > > > > Hi Gopal, > > > > My understanding is that the parameter below specifies the max size of 4G= B > for each contain. That seems to work for me > > > > > > mapreduce.map.memory.mb > > 4096 > > > > > > Now I am rather confused about the following parameters (for example > mapreduce.reduce versus mapreduce.map) and their correlation to each othe= r > > > > > > > > mapreduce.reduce.memory.mb > > 8192 > > > > > > > > mapreduce.map.java.opts > > -Xmx3072m > > > > > > > > mapreduce.reduce.java.opts > > -Xmx6144m > > > > > > Can you please verify if these settings are correct and how they relate t= o > each other? > > > > Thanks > > > > > > Mich Talebzadeh > > > > Sybase ASE 15 Gold Medal Award 2008 > > A Winning Strategy: Running the most Critical Financial Data on ASE 15 > > > http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-0919= 08.pdf > > Author of the books "A Practitioner=E2=80=99s Guide to Upgrading to Sybas= e ASE > 15", ISBN 978-0-9563693-0-7. > > co-author "Sybase Transact SQL Guidelines Best Practices", ISBN > 978-0-9759693-0-4 > > Publications due shortly: > > Complex Event Processing in Heterogeneous Environments, ISBN: > 978-0-9563693-3-8 > > Oracle and Sybase, Concepts and Contrasts, ISBN: 978-0-9563693-1-4, volum= e > one out shortly > > > > http://talebzadehmich.wordpress.com > > > > NOTE: The information in this email is proprietary and confidential. This > message is for the designated recipient only, if you are not the intended > recipient, you should destroy it immediately. Any information in this > message shall not be understood as given or endorsed by Peridale Technolo= gy > Ltd, its subsidiaries or their employees, unless expressly so stated. It = is > the responsibility of the recipient to ensure that this email is virus > free, therefore neither Peridale Ltd, its subsidiaries nor their employee= s > accept any responsibility. > > > > -----Original Message----- > From: Gopal Vijayaraghavan [mailto:gopal@hortonworks.com] On Behalf Of > Gopal Vijayaraghavan > Sent: 13 October 2015 20:55 > To: user@hive.apache.org > Cc: Mich Talebzadeh > Subject: Re: Container is running beyond physical memory limits > > > > > > > > > is running beyond physical memory limits. Current usage: 2.0 GB of 2 > > >GB physical memory used; 6.6 GB of 8 GB virtual memory used. Killing > > >container. > > > > You need to change the yarn.nodemanager.vmem-check-enabled=3Dfalse on > > *every* machine on your cluster & restart all NodeManagers. > > > > The VMEM check made a lot of sense in the 32 bit days when the CPU forced > a maximum of 4Gb of VMEM per process (even with PAE). > > > > Similarly it was a way to punish processes which swap out to disk, since > the pmem only tracks the actual RSS. > > > > In the large RAM 64bit world, vmem is not a significant issue yet - I > think the addressing limit is 128 TB per process. > > > > > > > > mapreduce.reduce.memory.mb > > > 4096 > > > > > ... > > > > > > mapreduce.reduce.java.opts > > > -Xmx6144m > > > > > > > That's the next failure point. 4Gb container with 6Gb limits. To produce > an immediate failure when checking configs, add > > > > -XX:+AlwaysPreTouch -XX:+UseNUMA > > > > to the java.opts. > > > > Cheers, > > Gopal > > --089e01176cadaa26e10522022c27 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

On Wed, Oct 14, 2015 at 1:42 AM, Mich Talebzadeh <mich@= peridale.co.uk> wrote:

Thank you all.

=C2=A0

Hi Gopal,

=C2=A0

My understanding is that the parameter below sp= ecifies the max size of 4GB for each contain. That seems to work for me<= /u>

=C2=A0

<property>

<name>= ;mapreduce.map.memory.mb</name>

<value>4096</= value>

</property>

=C2=A0<= /u>

Now I am rather confused about the following parameters (f= or example mapreduce.reduce versus= mapreduce.map) and their correlat= ion to each other

=C2=A0

=C2= =A0

<property>

<name>mapreduce.reduce.memory.mb= </name>

<value>8192</value><= /span>

</property>

=C2=A0

&= lt;property>

<name>mapreduce.map.java.opts</name= >

<value>-Xmx3072m</value>

<= ;/property>

=C2=A0

<property>=

<name>mapreduce.reduce.java.opts<= /name>

<value>-Xmx6144m</value>=

=C2=A0

=C2=A0<= u>

Mich Talebzadeh

=C2=A0

Sybase ASE 15 Gold Medal Award 2008

A Winning Strategy= : Running the most Critical Financial Data on ASE 15

http://login.sybase.com/files/Product_Over= views/ASE-Winning-Strategy-091908.pdf

Author of the= books "A Practitioner=E2=80=99s Guide to Upgrading to Sybase ASE 15&q= uot;, ISBN 978-0-9563693-0-7.

co-author "Sybase T= ransact SQL Guidelines Best Practices", ISBN 978-0-9759693-0-4<= u>

Publications due shortly:

Complex Event Pr= ocessing in Heterogeneous Environments, ISBN: 978-0-9563693-3-8

Oracle and Sybase, Concepts and Contrasts, ISBN: 978-0-9563693-1-4= , volume one out shortly

=C2=A0

http://taleb= zadehmich.wordpress.com

=C2=A0

= NOTE: The information in this email is proprietary and confidential. This m= essage is for the designated recipient only, if you are not the intended re= cipient, you should destroy it immediately. Any information in this message= shall not be understood as given or endorsed by Peridale Technology Ltd, i= ts subsidiaries or their employees, unless expressly so stated. It is the r= esponsibility of the recipient to ensure that this email is virus free, the= refore neither Peridale Ltd, its subsidiaries nor their employees accept an= y responsibility.

=C2=A0

<= div class=3D"h5">

-----Original Message-----
From= : Gopal Vijayaraghavan [mailto:gopal@hortonworks.com] On Behalf Of Gopal Vijayaraghavan=
Sent: 13 October 2015 20:55
To: user@hive.apache.org
Cc: Mich Talebzadeh <<= a href=3D"mailto:mich@peridale.co.uk" target=3D"_blank">mich@peridale.co.uk= >
Subject: Re: Container is running beyond physical memory limits=

=C2=A0

=C2=A0

= =C2=A0

> is running beyond physical memory limits. Current = usage: 2.0 GB of 2

>GB physical memory used; 6.6 GB= of 8 GB virtual memory used. Killing

>container.

=C2=A0

You need to change the yarn.n= odemanager.vmem-check-enabled=3Dfalse on

*every* machin= e on your cluster & restart all NodeManagers.

=C2=A0

The VMEM check made a lot of sense in the 32 bit days= when the CPU forced a maximum of 4Gb of VMEM per process (even with PAE).<= u>

=C2=A0

Similarly it was a way to pu= nish processes which swap out to disk, since the pmem only tracks the actua= l RSS.

=C2=A0

In the large RAM 64bi= t world, vmem is not a significant issue yet - I think the addressing limit= is 128 TB per process.

=C2=A0

>= <property>

> <name>mapreduce.reduce.mem= ory.mb</name>

> <value>4096</value>= ;

> </property>

...=

> <property>

=

> <name>mapreduce.reduce.java.opts</name>

> <value>-Xmx6144m</value>

> <= ;/property>

=C2=A0

That's the next failure point. 4Gb container with 6Gb limits. = To produce an immediate failure when checking configs, add

=C2=A0

-XX:+AlwaysPreTouch -XX:+UseNUMA

=C2=A0

to the java.opts.

<= u>=C2=A0

Cheers,

Gopal


--089e01176cadaa26e10522022c27--