incubator-drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Drawater <chris.drawa...@jdsu.com>
Subject Still unable to run a Distributed Drll Query...
Date Mon, 27 Oct 2014 11:33:14 GMT
We have 3 * Ubuntu 14.04 VM nodes each running a single 0.6 Drillbit , with 
1 of the VMs also running a Zookeeper 3.4.6 instance.

Each VM has an identical data directory structure on local persistent 
filesystem and hosts JSON files.

Zookeeper is aware of the 3 * Drillbits and 'select * from sys.drillbits' 
shows 3 Drillbits.

UDP Multicast for the VM subnet is supposedly enabled.



Using  Squirrl/JDBC driver on a Windows we can connect to Drill (vai the 
zookeeper instance) and run SQL queries against JSON files.

However the queries are only returning rows from the 'foreman' drillbit.



We have never managed to run a distributed query !



Indeed looking at the Drillbit logs we see :



Not a hint of any awareneess of other Drillbits

No 'fragments' mentioned in any plan apart from 'fragment_id : 0'



but we do see this (Client connected,via Zookeeper on Node 1,  to  Drillbit 
on Node 3) : 



2014-10-27 10:42:48,914 [dbc13c29-bbd8-4890-93fa-a8a98f4cc8fd:frag:0:0] 
ERROR o.a.drill.exec.ops.FragmentContext - Fragment Context received 
failure.

java.lang.RuntimeException: Failure while accessing Zookeeper

        at org.apache.drill.exec.store.sys.zk.ZkPStore.put
(ZkPStore.java:111) ~[drill-java-exec-0.6.0-incubating-rebuffed.jar:0.6.0-
incubating]

        at org.apache.drill.exec.work.foreman.QueryStatus.updateCache
(QueryStatus.java:125) ~[drill-java-exec-0.6.0-incubating-
rebuffed.jar:0.6.0-incubating]

        at org.apache.drill.exec.work.foreman.QueryStatus.update
(QueryStatus.java:119) ~[drill-java-exec-0.6.0-incubating-
rebuffed.jar:0.6.0-incubating]

        at org.apache.drill.exec.work.foreman.QueryManager.updateStatus
(QueryManager.java:173) ~[drill-java-exec-0.6.0-incubating-
rebuffed.jar:0.6.0-incubating]

        at org.apache.drill.exec.work.foreman.QueryManager.finished
(QueryManager.java:189) ~[drill-java-exec-0.6.0-incubating-
rebuffed.jar:0.6.0-incubating]

        at org.apache.drill.exec.work.foreman.QueryManager.statusUpdate
(QueryManager.java:162) ~[drill-java-exec-0.6.0-incubating-
rebuffed.jar:0.6.0-incubating]

        at 
org.apache.drill.exec.work.foreman.QueryManager$RootStatusHandler.statusChan
ge(QueryManager.java:284) ~[drill-java-exec-0.6.0-incubating-
rebuffed.jar:0.6.0-incubating]

        at 
org.apache.drill.exec.work.fragment.AbstractStatusReporter.finished
(AbstractStatusReporter.java:101) ~[drill-java-exec-0.6.0-incubating-
rebuffed.jar:0.6.0-incubating]

        at 
org.apache.drill.exec.work.fragment.AbstractStatusReporter.stateChanged
(AbstractStatusReporter.java:73) ~[drill-java-exec-0.6.0-incubating-
rebuffed.jar:0.6.0-incubating]

        at org.apache.drill.exec.work.fragment.FragmentExecutor.updateState
(FragmentExecutor.java:172) ~[drill-java-exec-0.6.0-incubating-
rebuffed.jar:0.6.0-incubating]

        at org.apache.drill.exec.work.fragment.FragmentExecutor.run
(FragmentExecutor.java:110) ~[drill-java-exec-0.6.0-incubating-
rebuffed.jar:0.6.0-incubating]

        at org.apache.drill.exec.work.WorkManager$RunnableWrapper.run
(WorkManager.java:250) [drill-java-exec-0.6.0-incubating-rebuffed.jar:0.6.0-
incubating]

        at java.util.concurrent.ThreadPoolExecutor.runWorker
(ThreadPoolExecutor.java:1145) [na:1.7.0_65]

        at java.util.concurrent.ThreadPoolExecutor$Worker.run
(ThreadPoolExecutor.java:615) [na:1.7.0_65]

        at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65]

Caused by: java.lang.InterruptedException: null

        at java.lang.Object.wait(Native Method) ~[na:1.7.0_65]

        at java.lang.Object.wait(Object.java:503) ~[na:1.7.0_65]

        at org.apache.zookeeper.ClientCnxn.submitRequest
(ClientCnxn.java:1309) ~[zookeeper-3.4.5.jar:3.4.5-1392090]

        at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1036) ~
[zookeeper-3.4.5.jar:3.4.5-1392090]

        at org.apache.curator.framework.imps.ExistsBuilderImpl$2.call
(ExistsBuilderImpl.java:172) ~[curator-framework-2.5.0.jar:na]

        at org.apache.curator.framework.imps.ExistsBuilderImpl$2.call
(ExistsBuilderImpl.java:161) ~[curator-framework-2.5.0.jar:na]

        at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107) ~
[curator-client-2.5.0.jar:na]

        at 
org.apache.curator.framework.imps.ExistsBuilderImpl.pathInForeground
(ExistsBuilderImpl.java:157) ~[curator-framework-2.5.0.jar:na]

        at org.apache.curator.framework.imps.ExistsBuilderImpl.forPath
(ExistsBuilderImpl.java:148) ~[curator-framework-2.5.0.jar:na]

        at org.apache.curator.framework.imps.ExistsBuilderImpl.forPath
(ExistsBuilderImpl.java:36) ~[curator-framework-2.5.0.jar:na]

        at org.apache.drill.exec.store.sys.zk.ZkPStore.put
(ZkPStore.java:104) ~[drill-java-exec-0.6.0-incubating-rebuffed.jar:0.6.0-
incubating]

        ... 14 common frames omitted



2014-10-27 10:42:48,926 [dbc13c29-bbd8-4890-93fa-a8a98f4cc8fd:frag:0:0] 
ERROR o.a.d.e.w.f.AbstractStatusReporter - Error 6f41051f-af65-4be8-9cce-
fe1895643d70: Failure while running fragment.

java.lang.InterruptedException: null

        at java.lang.Object.wait(Native Method) ~[na:1.7.0_65]

        at java.lang.Object.wait(Object.java:503) ~[na:1.7.0_65]

        at org.apache.zookeeper.ClientCnxn.submitRequest
(ClientCnxn.java:1309) ~[zookeeper-3.4.5.jar:3.4.5-1392090]

        at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1036) ~
[zookeeper-3.4.5.jar:3.4.5-1392090]

        at org.apache.curator.framework.imps.ExistsBuilderImpl$2.call
(ExistsBuilderImpl.java:172) ~[curator-framework-2.5.0.jar:na]

        at org.apache.curator.framework.imps.ExistsBuilderImpl$2.call
(ExistsBuilderImpl.java:161) ~[curator-framework-2.5.0.jar:na]

        at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107) ~
[curator-client-2.5.0.jar:na]

        at 
org.apache.curator.framework.imps.ExistsBuilderImpl.pathInForeground
(ExistsBuilderImpl.java:157) ~[curator-framework-2.5.0.jar:na]

        at org.apache.curator.framework.imps.ExistsBuilderImpl.forPath
(ExistsBuilderImpl.java:148) ~[curator-framework-2.5.0.jar:na]

        at org.apache.curator.framework.imps.ExistsBuilderImpl.forPath
(ExistsBuilderImpl.java:36) ~[curator-framework-2.5.0.jar:na]

        at org.apache.drill.exec.store.sys.zk.ZkPStore.put
(ZkPStore.java:104) ~[drill-java-exec-0.6.0-incubating-rebuffed.jar:0.6.0-
incubating]

        at org.apache.drill.exec.work.foreman.QueryStatus.updateCache
(QueryStatus.java:125) ~[drill-java-exec-0.6.0-incubating-
rebuffed.jar:0.6.0-incubating]

        at org.apache.drill.exec.work.foreman.QueryStatus.update
(QueryStatus.java:119) ~[drill-java-exec-0.6.0-incubating-
rebuffed.jar:0.6.0-incubating]

        at org.apache.drill.exec.work.foreman.QueryManager.updateStatus
(QueryManager.java:173) ~[drill-java-exec-0.6.0-incubating-
rebuffed.jar:0.6.0-incubating]

        at org.apache.drill.exec.work.foreman.QueryManager.finished
(QueryManager.java:189) ~[drill-java-exec-0.6.0-incubating-
rebuffed.jar:0.6.0-incubating]

        at org.apache.drill.exec.work.foreman.QueryManager.statusUpdate
(QueryManager.java:162) ~[drill-java-exec-0.6.0-incubating-
rebuffed.jar:0.6.0-incubating]

        at 
org.apache.drill.exec.work.foreman.QueryManager$RootStatusHandler.statusChan
ge(QueryManager.java:284) ~[drill-java-exec-0.6.0-incubating-
rebuffed.jar:0.6.0-incubating]

        at 
org.apache.drill.exec.work.fragment.AbstractStatusReporter.finished
(AbstractStatusReporter.java:101) ~[drill-java-exec-0.6.0-incubating-
rebuffed.jar:0.6.0-incubating]

        at 
org.apache.drill.exec.work.fragment.AbstractStatusReporter.stateChanged
(AbstractStatusReporter.java:73) ~[drill-java-exec-0.6.0-incubating-
rebuffed.jar:0.6.0-incubating]

        at org.apache.drill.exec.work.fragment.FragmentExecutor.updateState
(FragmentExecutor.java:172) ~[drill-java-exec-0.6.0-incubating-
rebuffed.jar:0.6.0-incubating]

        at org.apache.drill.exec.work.fragment.FragmentExecutor.run
(FragmentExecutor.java:110) ~[drill-java-exec-0.6.0-incubating-
rebuffed.jar:0.6.0-incubating]

        at org.apache.drill.exec.work.WorkManager$RunnableWrapper.run
(WorkManager.java:250) [drill-java-exec-0.6.0-incubating-rebuffed.jar:0.6.0-
incubating]

        at java.util.concurrent.ThreadPoolExecutor.runWorker
(ThreadPoolExecutor.java:1145) [na:1.7.0_65]

        at java.util.concurrent.ThreadPoolExecutor$Worker.run
(ThreadPoolExecutor.java:615) [na:1.7.0_65]

        at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65]



Does anyone have any ideas or pointers regarding this ? 



Also, I have a few questions...



1. When do the Drillbits become 'aware' of each other ?

2. Is there any Drill tracing that can switched on to reveal the   (lack 
of) communication between the Drillbits ?



Any help once again gratefully received.



Thanks,

    Chris






Mime
View raw message