hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nitay Joffe (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HBASE-1606) Remove zoo.cfg, put config options into hbase-site.xml
Date Thu, 02 Jul 2009 22:41:47 GMT

     [ https://issues.apache.org/jira/browse/HBASE-1606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Nitay Joffe updated HBASE-1606:

    Attachment: hbase-1606.patch

Enjoy zoo.cfg while you can boys. It's time to say goodbye.

In this patch:
- Remove zoo.cfg from conf/ and src/test.
- Remove zoo.cfg from build.xml and stargate build.xml jar targets.
- Add properties hbase.zookeeper.quorum, hbase.zookeeper.peerport, and hbase.zookeeper.leaderport.
Together, these form the server.X lines in the zoo.cfg.
- Add a property for every zoo.cfg option named hbase.zookeeper.property.KEY. Specifically,
this adds:
-- hbase.zookeeper.property.tickTime
-- hbase.zookeeper.property.initLimit
-- hbase.zookeeper.property.syncLimit
-- hbase.zookeeper.property.dataDir
-- hbase.zookeeper.property.clientPort
-- hbase.zookeeper.property.maxClientCnxns

- All of our ZooKeeper clients, which go through ZooKeeperWrapper, and our server wrapper,
HQuorumPeer, end up calling HQuorumPeer.makeZKProps() to get a Properties of the ZooKeeper
configuration. In that method, if there is a zoo.cfg in the classpath, simply load the config
from it. Otherwise, map each of the hbase.zookeeper.property.X options directly to the Properties
and create the server.X lines from the first three properties mentioned.

Note that this now means you can programatically change the ZooKeeper quorum used by setting
the "hbase.zookeeper.quorum" property and passing in your altered HBaseConfiguration object.
Along with this, there is no more static loading of a ZooKeeper config. Each ZooKeeperWrapper
does it on their own, so you can have multiple HTable or whatnot clients talking to different

> Remove zoo.cfg, put config options into hbase-site.xml
> ------------------------------------------------------
>                 Key: HBASE-1606
>                 URL: https://issues.apache.org/jira/browse/HBASE-1606
>             Project: Hadoop HBase
>          Issue Type: Sub-task
>    Affects Versions: 0.20.0
>            Reporter: Nitay Joffe
>            Assignee: Nitay Joffe
>             Fix For: 0.20.0
>         Attachments: hbase-1606.patch
> From HBASE-1551:
> Here's my current thinking:
>     * Move all of the ZooKeeper config paraments into hbase-*.xml using zookeeper.property.KEY
>     * Add a special property for the list of quorum servers, say zookeeper.quorum. This
option can default to "localhost".
>     * If there is a zoo.cfg present in the classpath, use its data above the zookeeper.property.KEY
>     * When we need to instantiate something to talk to ZooKeeper, we simply create a
new HBaseConfiguration and call some method on it e.g. toZooKeeperProperties().
>       This method will iterate through the zookeeper.property.KEY and turn each into
the appropriate ZooKeeper configurations (i.e. KEY=VALUE). It will generate
>       the server.X property from the zookeeper.quorum configuration option. As mentioned
above, if there is a zoo.cfg in the classpath, overwrite the data with its configuration.
>       This will return a Properties object that can be used to construct the appropriate
ZooKeeper config and start/talk to their servers.
>     * For start/stop management of full ZK quorum cluster, use something like my ZKServerTool
in the patch (modified of course) to do the parsing mentioned above and turn it
>       into a simple line-by-line list of quorum servers. As I do in this patch, the bin/zookeepers.sh
can then simply call bin/hbase o.a.h.h.z.ZKServerTool to get the list of hosts.
>       If you want something like a conf/zookeepers you can simply run ZKServerTool yourself.
> The benefits from all this are:
>     * One place for all ZK configuration. No duplicate setting of parameters.
>     * No more nasty zoo.cfg. Give the user what they're already used to, a single XML
config file.
>     * New user only need edit zookeeper.quorum to get full cluster.
>     * Programmable control of what ZK one is talking to.
> And some of Stack's comments:
> @nitay, I think you need to keep the hbase zookeeper config inside of an hbase namespace.
The Hadoop Configuation system is a floozy. It will go with anyone who calls load resource
on it pulling in their properties. I could see that out on a MR task, the Configuration could
have all kinds of pollution in it. Would suggest an hbase prefix - hbase.zookeeper prefix?
> @nitay on "if zoo.cfg in the CLASSPATH", that might work. We might want to try narrow
the places we look on the CLASSPATH. But lets start open and narrow later (we probably want
this to be open as possible at mo. until we learn more about the cloudera config.)
> @nitay on toZooKeeperProperties, do we have to expose that? Can't we just pass a HBaseConfiguation
to the HBase ZK Wrapper (I'm not up on latest dev here so this might be an off suggestion)

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message