incubator-drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mufy <mufeed.us...@gmail.com>
Subject Re: About Storage Plugin Instance Broadcast
Date Fri, 27 Jun 2014 07:09:03 GMT
Hello Anil,

Hmm... so the ZK will have the info. Agreed. But what about the statement
where it says that any new storage plugin added/updated will be broadcast
to all the nodes running drillbit service. I took that statement to mean -
the config file will be synchronized across all the nodes running the
drillbit service. Did I get it wrong?


---
Mufeed Usman
My LinkedIn <http://www.linkedin.com/pub/mufeed-usman/28/254/400> | My
Social Cause <http://www.vision2016.org.in/> | My Blogs : LiveJournal
<http://mufeed.livejournal.com>





On Fri, Jun 27, 2014 at 11:57 AM, AnilKumar B <akumarb2010@gmail.com> wrote:

> Hi Murfy,
>
> Based on my understanding, registered storage plugin info will be stored on
> ZK. As below.
>
> [zk: localhost:2181(CONNECTED) 10] ls /drill/sys.storage_plugins
> [hive, cp, dfs, hbase]
> [zk: localhost:2181(CONNECTED) 11] get /drill/sys.storage_plugins/hbase
> {
>   "type" : "hbase",
>   "config" : {
>     "hbase.zookeeper.quorum" : "localhost",
>     "hbase.zookeeper.property.clientPort" : "2181"
>   },
>   "enabled" : true
> }
> cZxid = 0x428
> ctime = Tue Jun 24 13:30:18 IST 2014
> mZxid = 0x450
> mtime = Tue Jun 24 13:31:41 IST 2014
> pZxid = 0x428
> cversion = 0
> dataVersion = 1
> aclVersion = 0
> ephemeralOwner = 0x0
> dataLength = 157
> numChildren = 0
> [zk: localhost:2181(CONNECTED) 12] get /drill/sys.storage_plugins/cp
> {
>   "type" : "file",
>   "enabled" : true,
>   "connection" : "classpath:///",
>   "workspaces" : null,
>   "formats" : null
> }
> cZxid = 0x426
> ctime = Tue Jun 24 13:30:18 IST 2014
> mZxid = 0x426
> mtime = Tue Jun 24 13:30:18 IST 2014
> pZxid = 0x426
> cversion = 0
> dataVersion = 0
> aclVersion = 0
> ephemeralOwner = 0x0
> dataLength = 118
> numChildren = 0
> [zk: localhost:2181(CONNECTED) 13]
>
> Thanks,
> B Anil Kumar.
>
> Thanks & Regards,
> B Anil Kumar.
>
>
> On Fri, Jun 27, 2014 at 11:12 AM, mufy <mufeed.usman@gmail.com> wrote:
>
> > Stated here
> >
> >
> https://cwiki.apache.org/confluence/display/DRILL/Connecting+Apache+Drill+to+Data+Sources
> > that,
> >
> > "When you add or update storage plugin instances on one node, Drill
> > broadcasts the information to all of the other Drill nodes so they have
> > identical storage plugin instances registered. You do not need to restart
> > any of the Drillbits when you add or update a storage plugin instance."
> >
> > Does that mean if I were to have the following on one node:
> > [root@mufeed-drill1 ~]# cat
> >
> >
> /opt/mapr/drill/drill-1.0.0/apache-drill-1.0.0-m2-incubating-SNAPSHOT/conf/storage-plugins.json
> > {
> >   "storage": {
> >
> >     dfs: {
> >       type: "file",
> >       connection: "maprfs:///"
> >     },
> >
> >     cp: {
> >       type: "file",
> >       connection: "classpath:///"
> >     },
> >
> >     cp1: {
> >       type: "file",
> >       connection: "classpath:///"
> >     },
> >       }
> > }
> >
> > The same content would get replicated to other nodes, on a file content
> > level? If yes, I do not see that happening. If not, please clarify.
> >
> > ---
> > Mufeed Usman
> > My LinkedIn <http://www.linkedin.com/pub/mufeed-usman/28/254/400> | My
> > Social Cause <http://www.vision2016.org.in/> | My Blogs : LiveJournal
> > <http://mufeed.livejournal.com>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message