incubator-drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Drawater <chris.drawa...@jdsu.com>
Subject Re: Distributed SQL on Drill question
Date Fri, 24 Oct 2014 10:33:35 GMT
Jason Altekruse <altekrusejason@...> writes:











> 



> Hi Chris,



> 



> Welcome to the drill community!



> 



> What are you currently using as a VM environment? Is it running on your 

own



> hardware or a cloud provider like AWS or Google Compute Engine? So far



> drill has required the nodes to be able to communicate using UDP multicast



> to sync some query information around the cluster for distributed queries.



> We realized that these providers do not support this level of network



> communication between nodes, so we have been trying to develop a



> workaround, but distributed queries will currently fail in these



> environments.



> 



> If this is not the case, a look through the logs for more detailed errors



> would be the most useful for debugging your specific issue.



> 



> Connecting to a specific node or a zk quorum should be the same in terms 

of



> allowing execution of distributed queries, both JDBC and ODBC connections



> will run distributed queries with locality information taken into account



> where available.



> 



> Distributed queries can be identified by exchanges inserted in the plan,



> this is where data is sent around to various nodes to split or merge the



> data when an operation is parallelized.



> 



> -Jason Altekruse











Thanks Jason.



We're currently using vSphere on a Windows 2010 host for dev/test purposes.



The VM network is bridged onto our internal(corporate) network and we've 

been told that multicast support is disabled by (corporate) default.... and 

that a multicast router would be required for our subnet...



Apparently multicast support has now been enabled but our distr SQL queries 

still only return data from a single Drillbit  and the logs show no 

communication between Drillbits... so maybe we still have a UDP multicast 

issue.



I guess the next step is to find some Linux UDP test utilities and/or write 

a simple Java UDP test application.



Chris 














Mime
View raw message