Return-Path: X-Original-To: apmail-incubator-drill-user-archive@minotaur.apache.org Delivered-To: apmail-incubator-drill-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 8D5F1179D7 for ; Mon, 27 Oct 2014 18:02:33 +0000 (UTC) Received: (qmail 87790 invoked by uid 500); 27 Oct 2014 18:02:33 -0000 Delivered-To: apmail-incubator-drill-user-archive@incubator.apache.org Received: (qmail 87731 invoked by uid 500); 27 Oct 2014 18:02:33 -0000 Mailing-List: contact drill-user-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: drill-user@incubator.apache.org Delivered-To: mailing list drill-user@incubator.apache.org Received: (qmail 87719 invoked by uid 99); 27 Oct 2014 18:02:32 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 27 Oct 2014 18:02:32 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of ted.dunning@gmail.com designates 209.85.192.177 as permitted sender) Received: from [209.85.192.177] (HELO mail-pd0-f177.google.com) (209.85.192.177) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 27 Oct 2014 18:02:27 +0000 Received: by mail-pd0-f177.google.com with SMTP id v10so6016123pde.8 for ; Mon, 27 Oct 2014 11:02:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:references:from:content-type:in-reply-to:message-id:date:to :content-transfer-encoding:mime-version; bh=f4Om5nzzpCfHNoH9SmcmEJseQiOa/T/5xhQT+jEWjSc=; b=lz9HmOtgDBWAXM+Qo7AxCZz2UzsMsequz2Ug0ooeHco5BxhFCKWux+r+w9cJckCxTn bWvptpCguOMBuqzboYz7Yd3PorP+oDfHsjsJ4CkgqSRqbGdjrurhCLD08KtlZZWf4Ax3 b0nmmrnsqYsBusdoEKw31K5d+hJSNTAd2Mn0KxP/XzQTpWKYvB17OVZWm0GHPc48uEpD OR+RhtPLlhMAXVFl5WJEynBeB6mWqh5JZyi36JmsckL4YKfRxufFvUwJblPdRsSM9Bas XRIytEdw4CO0DNgFwPsb/jBQrB5O4pFFyAUJWVb+eLzdeUVPtDCtuJ/Fs7O9WLBLPBKt HubQ== X-Received: by 10.70.33.73 with SMTP id p9mr25860852pdi.103.1414432927165; Mon, 27 Oct 2014 11:02:07 -0700 (PDT) Received: from [10.100.71.248] ([166.170.37.206]) by mx.google.com with ESMTPSA id cw5sm11455097pbc.9.2014.10.27.11.02.05 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 27 Oct 2014 11:02:06 -0700 (PDT) Subject: Re: Still unable to run a Distributed Drll Query... References: From: Ted Dunning Content-Type: text/plain; charset=us-ascii X-Mailer: iPhone Mail (11D201) In-Reply-To: Message-Id: <67ED15BE-42EF-4739-9E7D-E4104CEAEB7D@gmail.com> Date: Mon, 27 Oct 2014 11:02:01 -0700 To: "drill-user@incubator.apache.org" Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (1.0) X-Virus-Checked: Checked by ClamAV on apache.org I may have missed it as it went by, but what was the evidence that the zk qu= orum actually includes all the zookeeper nodes? This could be answered by e= xamination if the logs, but more definitive and simpler might be to configur= e to use only one zk node instead of three.=20 The rationale here is that if difference drillbits talked to different zk mo= des they could well have not known about each other. =20 Sent from my iPhone > On Oct 27, 2014, at 10:26, Chris Drawater wrote:= >=20 > Ramana Inukonda writes: >=20 >=20 >=20 >=20 >> Could you look at the zookeeper logs and see if there is any information >=20 >> there? Zookeeper logs should be at zk install location/ logs. There shoul= d >=20 >> be two files. A .log and .out. Please check both. >=20 >=20 >> Regards >=20 >> Ramana >=20 >=20 >=20 >=20 > Thanks Ramana. >=20 >=20 >=20 > We've now isolated our 3 * VMs onto their own private network... >=20 >=20 >=20 > Now we see the following in the DrillBit.log : >=20 >=20 >=20 >=20 >=20 > 2014-10-27 15:34:37,461 [d80e5b2c-3658-47ff-be30-fe884475feab:frag:0:0]=20= > WARN o.a.d.e.p.impl.SendingAccountor - Failure while waiting for send=20 > complete. >=20 > java.lang.InterruptedException: null >=20 > at=20 > java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInter= ru > ptibly(AbstractQueuedSynchronizer.java:996) ~[na:1.7.0_65] >=20 > at=20 > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterru= pt > ibly(AbstractQueuedSynchronizer.java:1303) ~[na:1.7.0_65] >=20 > at java.util.concurrent.Semaphore.acquire(Semaphore.java:472) ~ > [na:1.7.0_65] >=20 > at=20 > org.apache.drill.exec.physical.impl.SendingAccountor.waitForSendComplete > (SendingAccountor.java:44) ~[drill-java-exec-0.6.0-incubating-rebuffe >=20 > d.jar:0.6.0-incubating] >=20 > at org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.sto= p > (ScreenCreator.java:186) [drill-java-exec-0.6.0-incubating-rebuffed.jar:0.= 6. >=20 > 0-incubating] >=20 > at=20 > org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources > (FragmentExecutor.java:134) [drill-java-exec-0.6.0-incubating-rebuffed. >=20 > jar:0.6.0-incubating] >=20 > at org.apache.drill.exec.work.fragment.FragmentExecutor.run > (FragmentExecutor.java:109) [drill-java-exec-0.6.0-incubating- > rebuffed.jar:0.6.0-incu >=20 > bating] >=20 > at org.apache.drill.exec.work.WorkManager$RunnableWrapper.run > (WorkManager.java:250) [drill-java-exec-0.6.0-incubating-rebuffed.jar:0.6.= 0- > incubat >=20 > ing] >=20 > at java.util.concurrent.ThreadPoolExecutor.runWorker > (ThreadPoolExecutor.java:1145) [na:1.7.0_65] >=20 > at java.util.concurrent.ThreadPoolExecutor$Worker.run > (ThreadPoolExecutor.java:615) [na:1.7.0_65] >=20 > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65] >=20 >=20 >=20 > but no corresponding errors in the Zookeeper logs... >=20 >=20 >=20 > Chris >=20 >=20 >=20 >=20 >=20 >=20 >=20