incubator-drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aditya <adityakish...@gmail.com>
Subject Re: Recent Commits - things to watch out for if you're upgrading
Date Wed, 11 Jun 2014 23:45:07 GMT
"org.apache.drill.exec.store.sys.zk.ZkPStoreProvider" is default
<https://github.com/apache/incubator-drill/blob/master/exec/java-exec/src/main/resources/drill-module.conf#L98>
for distributed mode and you need not specify it explicitly.

If you want to use HBase storage instead, you can specify "
org.apache.drill.exec.store.hbase.config.HBasePStoreProvider
<https://github.com/apache/incubator-drill/blob/master/contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/config/HBasePStoreProvider.java>"
as the storage provider as follow

  sys.store.provider: {
>     class: "org.apache.drill.exec.store.hbase.config.HBasePStoreProvider",
>     hbase: {
>       tableName : "drill_store",
>       config : {
>         "hbase.zookeeper.quorum" : "localhost",
>         "hbase.zookeeper.property.clientPort" : 2181
>       }
>     }
>   }
>

The provider class is part of "drill-storage-hbase" project.



On Wed, Jun 11, 2014 at 4:23 PM, Xiao Meng <xiaom.cs@gmail.com> wrote:

> Hi,
>
> I've tried it both in embedded mode and distributed mode and works fine for
> me now.
>
> For the distributed mode, we need to set the sys.store.provider as
> zookeeper
>
> sys.store.provider: {
>  class: "org.apache.drill.exec.store.sys.zk.ZkPStoreProvider"
> },
>
> One question:
>
> > In daemon/distributed mode, you can also store them in HBase if you have
> that available as part of your cluster.
> So do we need to write the plugin for HBase? I did not see any class in
> exec.store.sys.
>
>
> Hope it will help for the other people.
>
> BTW, can someone give the edit permission on the wiki? I can put this into
> wiki. My user name is xiaom.
>
> Thanks,
>
> Xiao
>
>
> On Wed, Jun 11, 2014 at 12:36 PM, Jacques Nadeau <jacques@apache.org>
> wrote:
>
> > Hey Guys,
> >
> > I've had a couple of questions about the commits that went in last night.
> >  Some things changed in configuration that people should be aware of.
>  They
> > are as follows:
> >
> > - drill-override.conf is basically empty.  A sample drill-override is
> > available as well to see what some settings are that are available.  You
> > should move to this setup only migrate any settings that you have changed
> > from previous defaults.
> > - If you use your old settings, you'll likely to encounter a Zookeeper
> > exception.  This is because the drill.exec.zk.root no longer supports
> have
> > a leading slash (e.g. /drill).  To make it work, you need to either
> follow
> > the recommendation above or remove the leading slash.
> > - views, storage plugins and system settings are persisted across
> drillbit
> > restarts.
> > - view are persisted in writable workspaces (default one is dfs.tmp) as
> > json files called viewname.view.drill.
> > - Storage plugins and other drill settings are stored in a Drill PStores
> (a
> > persistent store for settings information).  In embedded mode they are
> > persisted to the local file system.  By default, in daemon/distributed
> > mode, they are stored in zookeeper.  In daemon/distributed mode, you can
> > also store them in HBase if you have that available as part of your
> > cluster.
> > - In order to configure storage plugins, you need to use the web ui at
> > http://localhost:8047
> > - By default, Drill is initially populated with
> > bootstrap-storage-plugins.json in your classpath (Drill packages one in
> one
> > of the jars or you can put your own earlier in the classpath).  Once the
> > first node comes up and populates the storage plugin configuration, Drill
> > no longer uses or considers this file.
> >
> >
> > Let me know if there are other questions or issues that come up.  Also,
> be
> > sure to do a full clean build with this so you don't have any old/new
> file
> > conflicts.
> >
> > thanks,
> > Jacques
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message