drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Timothy Chen <tnac...@gmail.com>
Subject Re: Deploying Drill in a clustered Environment
Date Mon, 16 Jun 2014 21:07:43 GMT
Hi Terry,

I believe dfs is usually referring to local file system, but they're
mostly just aliases under storage-plugins.json that refers to the
actual storage setting.

You can refer here for more information how to configure them:
https://cwiki.apache.org/confluence/display/DRILL/Connecting+Apache+Drill+to+Data+Sources

And yes for zk hosts, it's host:port seperated by commas, ie:
localhost:2181,localhost:2191

Tim

On Mon, Jun 16, 2014 at 2:00 PM, Terry Healy <thealy@bnl.gov> wrote:
> Thanks Tim.
>
> I'm starting with 6 nodes. Got as far as trying to registering file systems,
> but when I start Drill, all the storage plugins fail.
>
> I'm most interested to get HDFS up; is that 'dfs' or 'hadoop_simple'?
>
> I have hBase running on the cluster, but I see this error:
>
>  DEBUG o.a.d.e.s.h.HBaseStoragePluginConfig - Initializing HBase
> StoragePlugin configuration with zookeeper quorum 'localhost', port '2181'.
>
> So I assume I have to set it to my 3 active Zookeepers, in the config here:
>
> "hbase.zookeeper.quorum" : "localhost",
>
> So just hostnames separated by commas?
>
>
>
> Thanks for your help.
>
> -Terry
>
>
> On 06/16/2014 04:04 PM, Timothy Chen wrote:
>>
>> Hi Terry,
>>
>> You can start testing even with one node if you like, but ofcourse to
>> gain more power for parallelizing you want more nodes which is what
>> Drill is designed for.
>>
>> Drill can pull from HDFS from the configured HDFS information as part
>> of the storage engine config. More info about that should be available
>> in our wiki.
>>
>> Tim
>>
>>
>>
>> On Mon, Jun 16, 2014 at 12:41 PM, Terry Healy <thealy@bnl.gov> wrote:
>>>
>>> Hi-
>>>
>>> Is it necessary to install Drill on all the nodes of an HDFS cluster, or
>>> is
>>> it OK to start testing with a few?
>>>
>>> Thanks
>>>
>

Mime
View raw message