drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Timothy Chen <tnac...@gmail.com>
Subject Re: Distributed SQL on Drill question
Date Wed, 22 Oct 2014 21:58:20 GMT
Hi Chris,

It should work as you described, so can you tell us what error message
you're getting?

Tim

On Wed, Oct 22, 2014 at 2:38 AM, Chris Drawater <Chris.Drawater@jdsu.com>
wrote:

>
>
> Hi,
>
>
>
> We have started to evaluate Apache Drill.
>
>
>
> Using  multiple nodes/VMs  we wish to stream JSON data (via Apache Storm)
> to  persistent local filesystem (with a consistent dir structure so we can
> define various storage plugins) and then use Drill to run distributed SQL
> queries across these JSON files.
>
> If at all possible we don't wish to install Hadoop HDFS/Hbase/Hive.
>
>
>
>
>
> SQL execution would be hopefully be via ODBC and/or JDBC.
>
>
>
> So far , embedded drill experiments work just great but the data processed
> is local to the sqlline.
>
> Using SQL against JSON works really well.
>
>
>
> Unfortunately,  experiments to run distributed SQL queries have (so far)
> not been successful.
>
> We have tried
>
> ·         a 3 node VM based system  with a 3 node zookeeper quorum + 1
> drillbit per node
>
> ·         a 3 node VM based system  with a single zookeeper instance
> (covering all 3 nodes)   + again 1 drillbit per node
>
> and 'select * from sys.drillbits' from sqlline outputs all 3 drill bits in
> both  case.   *Likewise, zookeeper confirms the existence of the drill
> cluster.*
>
> But we’ve not managed to run a distributed query.
>
>
>
> We have tested against both the 0.5 release and a 0.6 build on 64 bit
> Ubuntu 14.04.
>
>
>
> So far, we can only connect via ODBC to a specific drillbit.
>
> We cannot get the JDBC driver (using squirrel) to work.
>
> Also ODBC to a zookeeper quorum  doesn't appear to work.  * But we have
> tested client access using telnet IP_ADDR 2181  and that's OK.*
>
>
>
> So my questions are :
>
>
>
> ·         Would we expect to be able to run a distr SQL query by
> connecting  via JDBC direct to specific drillbit ?
>
> ·         Would we expect to be able to run a distr SQL query by
> connecting  via ODBC direct to specific drillbit ?
>
> ·         Would we expect to be able to run a distr SQL query by
> connecting  via JDBC via a zookeeper quorum connection ?
>
> ·         Would we expect to be able to run a distr SQL query by
> connecting  via ODBC via a zookeeper quorum connection ?
>
> ·         How can we identify the use of multiple nodes due to a
> distributed  SQL query    via explain plan output or the JSON QEP ?
>
> ·         Any ideas or issues why we can't connect via the a zookeeper
> quorum connection ?
>
>
>
> Any help or insights you can give would be most appreciated.
>
>
>
> Thanks.
>
>    Chris
>
>
>
> Chris Drawater
>
> Database Architect
>
> *[image: Description: AriesoA-JDSU-Mobility-Solution_logo 300px wide]*
> <http://www.arieso.com/>
>
> Office +44 1635 232470  |  Fax +44 1635 232471
>
> Email chris.drawater@jdsu.com  |  Web www.arieso.com
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message