drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Drawater <chris.drawa...@jdsu.com>
Subject Test Case for Inability to run a Distributed Drill Query (on JSON files on FS).
Date Wed, 29 Oct 2014 09:30:27 GMT
Hi Ramana,





We've isolated the problem!   Below is our simple test case whereby a SQL 
query only runs local and not distributed.



Our single zookeeper instance is running only on node Drill1  amd the 
foreman drillbit is running on Drill2.



SQL -->

	use dfs;

	select xbin,ybin,numberOfConnections  from 
`/geo/appdata/rawdata/2014/SameNamedata.json`;



where SameNamedata.json is present on  nodes : Drill1, Drill2 and Drill3,  
and is of the form -->



{ "NumberOfConnections": 1, "StartTime": "2014-10-
02T14:30:00Z", "EndTime": "2014-10-02T14:45:00Z", "XBin": 566150, "YBin": 
3270100 }

{ "NumberOfUniqueSubscribers": 2, "NumberOfConnections": 
2, "StartTime": "2014-10-02T14:30:00Z", "EndTime": "2014-10-
02T14:45:00Z", "XBin": 571350, "YBin": 3268100, "AverageSnr": -
6.2870720195528573 }



On the 'foreman' node (Drill2) ,  drillbit.log  reveals the following 
trace -->



2014-10-28 10:55:06,883 [UserServer-1] DEBUG 
o.a.drill.exec.rpc.user.UserServer - Received query to run.  Returning 
query handle.

2014-10-28 10:55:06,893 [WorkManager Event Thread] DEBUG 
o.apache.drill.exec.work.WorkManager - Starting pending task 
org.apache.drill.exec.work.WorkManager$RunnableWrapper@bec4752

2014-10-28 10:55:06,911 [091c7c1d-fb6b-4685-9f1d-04f08afd1053:foreman] 
DEBUG o.a.d.e.s.h.HBaseStoragePluginConfig - Initializing HBase 
StoragePlugin configuration with zookeeper quorum 'localhost', port '2181'.

2014-10-28 10:55:06,912 [091c7c1d-fb6b-4685-9f1d-04f08afd1053:foreman] 
DEBUG o.a.drill.exec.store.SchemaFactory - Took 17 ms to register schemas.

2014-10-28 10:55:06,953 [091c7c1d-fb6b-4685-9f1d-04f08afd1053:foreman] 
DEBUG o.a.d.e.p.s.h.DefaultSqlHandler - Optiq Logical :

ProjectRel(xbin=[$1], ybin=[$2], numberOfConnections=[$3]): rowcount = 
100.0, cumulative cost = {200.0 rows, 401.0 cpu, 0.0 io, 0.0 network, 0.0 
memory}, id = 1259

  EnumerableTableAccessRel(table=
[[dfs, /geo/appdata/rawdata/2014/SameNamedata.json]]): rowcount = 100.0, 
cumulative cost = {100.0 rows, 101.0 cpu, 0.0 io, 0.0 network, 0.0 memory}, 
id = 1256



The line -->

	2014-10-28 10:55:06,911 [091c7c1d-fb6b-4685-9f1d-
04f08afd1053:foreman] DEBUG o.a.d.e.s.h.HBaseStoragePluginConfig - 
Initializing HBase StoragePlugin configuration with zookeeper 
quorum 'localhost', port '2181'.

may be the key.



We're not using HBase and their is no Zookeeper instance running local on 
Drill2 !!



May be a bug ?  (Although it might still be our configuration I suppose).





Cheers,

   Chris


Mime
View raw message