Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 5DEFD200BB1 for ; Thu, 3 Nov 2016 21:38:25 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 5C7B3160AFF; Thu, 3 Nov 2016 20:38:25 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 55173160AE5 for ; Thu, 3 Nov 2016 21:38:24 +0100 (CET) Received: (qmail 60217 invoked by uid 500); 3 Nov 2016 20:38:23 -0000 Mailing-List: contact user-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hive.apache.org Delivered-To: mailing list user@hive.apache.org Received: (qmail 60207 invoked by uid 99); 3 Nov 2016 20:38:23 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Nov 2016 20:38:23 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 911D218009D for ; Thu, 3 Nov 2016 20:38:22 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.779 X-Spam-Level: ** X-Spam-Status: No, score=2.779 tagged_above=-999 required=6.31 tests=[FSL_HELO_BARE_IP_2=1.499, HTML_MESSAGE=2, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id hMtANHZFFoE2 for ; Thu, 3 Nov 2016 20:38:20 +0000 (UTC) Received: from relayvx11c.securemail.intermedia.net (relayvx11c.securemail.intermedia.net [64.78.52.185]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 03E995F5A1 for ; Thu, 3 Nov 2016 20:38:19 +0000 (UTC) Received: from emg-ca-1-1.securemail.intermedia.net (localhost [127.0.0.1]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by emg-ca-1-1.localdomain (Postfix) with ESMTPS id 7303B53F43 for ; Thu, 3 Nov 2016 13:38:18 -0700 (PDT) Subject: Re: Hive/Tez local mode running out of memory MIME-Version: 1.0 x-echoworx-msg-id: 4515a896-841e-40d9-ab91-31975acb80ed x-echoworx-emg-received: Thu, 3 Nov 2016 13:38:18.374 -0700 x-echoworx-message-code-hashed: 176eec26620eff975b92a51d05b41f6562cd75b4471848422ca304b6167d3a6f x-echoworx-action: delivered Received: from 10.254.155.15 ([10.254.155.15]) by emg-ca-1-1 (JAMES SMTP Server 2.3.2) with SMTP ID 834 for ; Thu, 3 Nov 2016 13:38:18 -0700 (PDT) Received: from MBX080-W3-CO-1.exch080.serverpod.net (unknown [10.224.117.52]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by emg-ca-1-1.localdomain (Postfix) with ESMTPS id 1EA2053F43 for ; Thu, 3 Nov 2016 13:38:18 -0700 (PDT) Received: from MBX080-W3-CO-2.exch080.serverpod.net (10.224.117.53) by MBX080-W3-CO-1.exch080.serverpod.net (10.224.117.52) with Microsoft SMTP Server (TLS) id 15.0.1178.4; Thu, 3 Nov 2016 13:38:16 -0700 Received: from MBX080-W3-CO-2.exch080.serverpod.net ([10.224.117.53]) by mbx080-w3-co-2.exch080.serverpod.net ([10.224.117.53]) with mapi id 15.00.1178.000; Thu, 3 Nov 2016 13:38:17 -0700 From: Prasanth Jayachandran To: "user@hive.apache.org" Thread-Topic: Hive/Tez local mode running out of memory Thread-Index: AQHSMZ6LznKKH5HxS0e016CFduA+9aDAA3wsgADuJwCAA/iTgIAABgGAgABCAACAAawEAIAACrWAgAAo+oCAAAN9AIABIRuA Date: Thu, 3 Nov 2016 20:38:16 +0000 Message-ID: <5FE260A0-23AC-4524-9651-B29CDC81374F@hortonworks.com> References: <2C99667D-505A-4C77-93C2-A11253A5D341@hortonworks.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [192.175.27.10] x-source-routing-agent: Processed Content-Type: multipart/alternative; boundary="_000_5FE260A023AC45249651B29CDC81374Fhortonworkscom_" archived-at: Thu, 03 Nov 2016 20:38:25 -0000 --_000_5FE260A023AC45249651B29CDC81374Fhortonworkscom_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Premal This is the problematic line https://gist.github.com/premal/d054d4cc0ed00efdf60351ca2517db3d#file-hive-l= og4j2-properties-L23 Can you trying changing ALL to INFO or WARN and see if that helps? Thanks Prasanth On Nov 2, 2016, at 8:23 PM, Premal Shah > wrote: Sorry, here's the link - https://gist.github.com/premal/d054d4cc0ed00efdf60351ca25= 17db3d On Wed, Nov 2, 2016 at 8:11 PM, Premal Shah > wrote: Hi Prasanth, Here's a link to the hive log4js properties file. We are on Hive 2.0.1. We cannot use 2.1.0 because MSCK REPAIR TABLE is broken on it. Thanx. On Wed, Nov 2, 2016 at 5:44 PM, Prasanth Jayachandran > wrote: This looks related to https://issues.apache.org/jira/browse/HIVE-11751 The debug strings gets generated but gets filtered. Can you share your log4= j2 properties file? What version of hive are you using? Thanks Prasanth On Nov 2, 2016, at 5:06 PM, Premal Shah > wrote: Hi Gopal, The first thing I did was to look in the logs. Could not find anything even= after setting the log level to DEBUG. Any ideas will be appreciated. On Tue, Nov 1, 2016 at 3:34 PM, Gopal Vijayaraghavan > wrote: > I am not sure what is going on here. You can check the /tmp/$USER/hive.log and see what's happening in detail. Cheers, Gopal -- Regards, Premal Shah. -- Regards, Premal Shah. -- Regards, Premal Shah. --_000_5FE260A023AC45249651B29CDC81374Fhortonworkscom_ Content-Type: text/html; charset="us-ascii" Content-ID: <874F1381ABCBCE4685332660CD017E2D@exch080.serverpod.net> Content-Transfer-Encoding: quoted-printable Hi Premal

This is the problematic line
https://gist.gi= thub.com/premal/d054d4cc0ed00efdf60351ca2517db3d#file-hive-log4j2-propertie= s-L23

Can you trying changing ALL to INFO or WARN and see if that= helps?

Thanks
Prasanth


On Nov 2, 2016, at 8:23 PM, Premal Shah <premal.j.shah@gmail.com> wrot= e:

Sorry,
here's the link - https://gist.github.com/= premal/d054d4cc0ed00efdf60351ca2517db3d


On Wed, Nov 2, 2016 at 8:11 PM, Premal Shah <premal.j.shah@gmail.com> wrote:
Hi Prasanth,
Here's a link to the hive log4js properties file.

We are on Hive 2.0.1. 

We cannot use 2.1.0 because MSCK REPAIR TABLE is broken on = it.

Thanx.


On Wed, Nov 2, 2016 at 5:44 PM, Prasanth Jayacha= ndran <pjayachandran@hortonworks.com> wrote:<= br class=3D"">
This looks related to <= a href=3D"https://issues.apache.org/jira/browse/HIVE-11751" target=3D"_blan= k" class=3D"">https://issues.apache.org/jira/browse/HIVE-11= 751

The debug strings gets generated but gets filtered. Can you= share your log4j2 properties file?

What version of hive are you using? 

Thanks
Prasanth

On Nov 2, 2016, at 5:06 PM, Premal Shah <premal.j.shah@gmai= l.com> wrote:

Hi Gopal,
The first thing I did was to look in the logs. Could not fi= nd anything even after setting the log level to DEBUG. 
Any ideas will be appreciated. 



On Tue, Nov 1, 2016 at 3:34 PM, Gopal Vijayaragh= avan <gopal= v@apache.org> wrote:

> I am not sure what is going on= here. 

You can check the /tmp/$USER/hive.log and see what's happening in de= tail.

 

Cheers,

Gopal




--
Regards,
Premal Shah.




--
Regards,
Premal Shah.



--
Regards,<= br class=3D""> Premal Shah.

--_000_5FE260A023AC45249651B29CDC81374Fhortonworkscom_--