Return-Path: X-Original-To: apmail-incubator-drill-user-archive@minotaur.apache.org Delivered-To: apmail-incubator-drill-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3115A177AA for ; Fri, 31 Oct 2014 06:11:47 +0000 (UTC) Received: (qmail 56254 invoked by uid 500); 31 Oct 2014 06:11:46 -0000 Delivered-To: apmail-incubator-drill-user-archive@incubator.apache.org Received: (qmail 56198 invoked by uid 500); 31 Oct 2014 06:11:46 -0000 Mailing-List: contact drill-user-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: drill-user@incubator.apache.org Delivered-To: mailing list drill-user@incubator.apache.org Received: (qmail 56187 invoked by uid 99); 31 Oct 2014 06:11:46 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 31 Oct 2014 06:11:46 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of tridib.samanta@live.com designates 65.54.190.203 as permitted sender) Received: from [65.54.190.203] (HELO BAY004-OMC4S1.hotmail.com) (65.54.190.203) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 31 Oct 2014 06:11:39 +0000 Received: from BAY177-W14 ([65.54.190.199]) by BAY004-OMC4S1.hotmail.com over TLS secured channel with Microsoft SMTPSVC(7.5.7601.22751); Thu, 30 Oct 2014 23:11:15 -0700 X-TMN: [Aw20WWl8+jCbBQYx6BktEUsLF5pwiBli] X-Originating-Email: [tridib.samanta@live.com] Message-ID: Content-Type: multipart/alternative; boundary="_7dca677b-ab3a-4c60-a463-27aa5529aff9_" From: Tridib Samanta To: "drill-user@incubator.apache.org" Subject: distinct stuck after some time Date: Thu, 30 Oct 2014 23:11:15 -0700 Importance: Normal MIME-Version: 1.0 X-OriginalArrivalTime: 31 Oct 2014 06:11:15.0886 (UTC) FILETIME=[7A9374E0:01CFF4D1] X-Virus-Checked: Checked by ClamAV on apache.org --_7dca677b-ab3a-4c60-a463-27aa5529aff9_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi Experts=2C I am running a distinct query on parquet. I stuck after some time. Here is = the log. I have allocated Java direct memory 32GB and Heap size 24GB. There= are around 50 million record in parquet file. =20 uckets in hash table =3D 524288. 2014-10-31 02:06:09=2C717 [b54b8800-910c-4a01-a052-533def304e84:frag:2:6] D= EBUG o.a.d.e.p.impl.common.HashTable - Rehashing entries within the batch: = 61=3B batchStartIdx =3D 3997696=2C total numBuckets in hash table =3D 10485= 76. 2014-10-31 02:06:09=2C718 [b54b8800-910c-4a01-a052-533def304e84:frag:2:1] D= EBUG o.a.d.e.p.impl.common.HashTable - Put into hash table failed .. Retryi= ng with new batch holder... 2014-10-31 02:06:09=2C718 [b54b8800-910c-4a01-a052-533def304e84:frag:2:4] D= EBUG o.a.d.e.p.impl.common.HashTable - Put into hash table failed .. Retryi= ng with new batch holder... 2014-10-31 02:06:09=2C718 [b54b8800-910c-4a01-a052-533def304e84:frag:2:0] D= EBUG o.a.d.e.p.impl.common.HashTable - Put into hash table failed .. Retryi= ng with new batch holder... 2014-10-31 02:06:09=2C719 [b54b8800-910c-4a01-a052-533def304e84:frag:2:2] D= EBUG o.a.d.e.p.impl.common.HashTable - Put into hash table failed .. Retryi= ng with new batch holder... 2014-10-31 02:06:09=2C724 [b54b8800-910c-4a01-a052-533def304e84:frag:2:9] D= EBUG o.a.d.e.p.impl.common.HashTable - Put into hash table failed .. Retryi= ng with new batch holder... 2014-10-31 02:06:09=2C725 [b54b8800-910c-4a01-a052-533def304e84:frag:2:10] = DEBUG o.a.d.e.p.impl.common.HashTable - Put into hash table failed .. Retry= ing with new batch holder... 2014-10-31 02:06:09=2C730 [b54b8800-910c-4a01-a052-533def304e84:frag:2:8] D= EBUG o.a.d.e.p.impl.common.HashTable - Put into hash table failed .. Retryi= ng with new batch holder... 2014-10-31 02:06:09=2C739 [b54b8800-910c-4a01-a052-533def304e84:frag:2:5] D= EBUG o.a.d.e.p.impl.common.HashTable - Put into hash table failed .. Retryi= ng with new batch holder... =20 Any pointer? =20 Thanks & regards Tridib = --_7dca677b-ab3a-4c60-a463-27aa5529aff9_--