hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rita <rmorgan...@gmail.com>
Subject Re: zookeeper quorum verification
Date Sun, 04 Dec 2011 15:09:23 GMT
To add,

The correct way to setup your client/application is like this


<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
  <property>
    <name>hbase.zookeeper.quorum</name>
    <value>roundrobindnsdnsentry</value>
    <description>The directory shared by region servers.
    </description>
  </property>
</configuration>


Configuration config = HBaseConfiguration.create();
config.set("hbase.zookeeper.quorum", "roundrobindnsentry");  // Here
we are running zookeeper locally


On Sun, Dec 4, 2011 at 10:03 AM, Rita <rmorgan466@gmail.com> wrote:

> Thanks for the nice responses and advice.
>
> To sum up this thread
>
> This will not work,
>
> $ host roundrobindnsentry
> roundrobindnsentry has address 192.168.0.2
> roundrobindnsentry has address 192.168.0.5
> roundrobindnsentry has address 192.168.0.6
>
>
> $ host roundrobindnsentry
> roundrobindnsentry has address 192.168.0.5
> roundrobindnsentry has address 192.168.0.6
> roundrobindnsentry has address 192.168.0.2
>
>
> ├Źn your hbase-site.xml
>
> <property>
>       <name>hbase.zookeeper.quorum</name>
>       <value>roundrobindnsentry</value>
>       </description>
>     </property>
>
>
> This will not work because when you start up the hbase cluster it will
> complain the zookeeper is already running. Basically it will get confused
> because of the round robin DNS entries.
>
> However,  this WILL work if in your application you set, roundrobingdns
> entry.
>
>
>
>
>
>
>
> On Sat, Dec 3, 2011 at 8:09 PM, Shrijeet Paliwal <shrijeet@rocketfuel.com>wrote:
>
>> >>Now any HBase client that needs to connect to zookeeper, can talk to
>> quorum via "zookeeper-quorum.example.com"
>>
>> Must add the reason above works is zookeeper uses
>>  InetAddress.getAllByName
>> <
>> http://docs.oracle.com/javase/1.3/docs/api/java/net/InetAddress.html#getAllByName(java.lang.String)
>> >to
>> build the server address list
>>
>> On Sat, Dec 3, 2011 at 5:06 PM, Shrijeet Paliwal <shrijeet@rocketfuel.com
>> >wrote:
>>
>> > Suraj,
>> > Combine what J-D & David said. For example in our case
>> >
>> >    1. We let HBase manage zookeeper
>> >    2. Set the config in hbase-site.xml the way David mentioned
>> >    :
>> <value>0.zookeeper,1.zookeeper,2.zookeeper,3.zookeeper,4.zookeeper</value>
>> >    3. Also have a DNS entry "zookeeper-quorum.example.com" with
>> multiple
>> >    IPs (all mentioned above) resolving to this entry
>> >    4. Now any HBase client that needs to connect to zookeeper, can talk
>> >    to quorum via "zookeeper-quorum.example.com"
>> >
>> > One might think why we cant have hbase-site use ""
>> > zookeeper-quorum.example.com" as well. My understanding is when you
>> start
>> > a zookeeper daemon on a server, the hbase managed zookeeper service will
>> > fetch the the default host name using hadoop's DNS.getDefaultHost
>> method.
>> > The hostname returned may or may not find a match in the server list you
>> > mentioned in hbase-site.xml. If does not match you get the error
>> mentioned
>> > here http://wiki.apache.org/hadoop/Hbase/Troubleshooting#A9.
>> >
>> > For example, in our case zookeeper host has two hostnames associated
>> with
>> > default ethernet interface. First name is host.domain.com and second is
>> > the special DNS entry I talked about in (4). The DNS.getDefaultHost
>> returns
>> > host.domain.com and zookeeper fails to start.
>> >
>> > I would be interested if some one has a different way of handling the
>> > situation I described.
>> >
>> > On Sat, Dec 3, 2011 at 4:45 PM, Suraj Varma <svarma.ng@gmail.com>
>> wrote:
>> >
>> >> Yes - this makes sense. But, I thought what Rita suggested was a
>> >> single appquorum dns entry ... which was surprising.
>> >>
>> >> Hence my question.
>> >> --Suraj
>> >>
>> >> On Sat, Dec 3, 2011 at 10:40 AM, Dave Barr <dave.barr@gmail.com>
>> wrote:
>> >> > What we do is make N.zookeeper.$DC.$DOMAIN entries in DNS.  We have
>> >> > one ZK cluster per DC.  Our configs then just point to
>> >> >
>> >> >
>> >>
>> <value>0.zookeeper,1.zookeeper,2.zookeeper,3.zookeeper,4.zookeeper</value>
>> >> >
>> >> > --Dave
>> >> >
>> >> > On Sat, Dec 3, 2011 at 6:15 AM, Suraj Varma <svarma.ng@gmail.com>
>> >> wrote:
>> >> >> J-D:
>> >> >> Did you mean that a _single_ dns entry returns all five ips
>> belonging
>> >> >> to individual zk nodes?
>> >> >>
>> >> >> Is this used only by "clients" ... or even within the cluster?
>> >> >> And ... the zk nodes self-identify by IP ... and is this how region
>> >> >> server nodes reach out specifically to the "leader" zk node?
>> >> >> --Suraj
>> >> >>
>> >> >>
>> >> >> On Wed, Nov 30, 2011 at 4:14 PM, Jean-Daniel Cryans <
>> >> jdcryans@apache.org> wrote:
>> >> >>> It's pretty much what we do, works well.
>> >> >>>
>> >> >>> J-D
>> >> >>>
>> >> >>> On Wed, Nov 30, 2011 at 3:49 PM, Rita <rmorgan466@gmail.com>
>> wrote:
>> >> >>>> Hello,
>> >> >>>>
>> >> >>>>
>> >> >>>> Previously, I assigned 5 servers as part of the zookeeper
quorum.
>> >> >>>> Everything works fine but I was hard coding these 5 servers
>> >> everywhere and
>> >> >>>> I was thinking of creating a dns entry called appquorum
which will
>> >> always
>> >> >>>> return these 5 servers IPs.
>> >> >>>>
>> >> >>>> Any thoughts about this?
>> >> >>>>
>> >> >>>>
>> >> >>>>
>> >> >>>> --
>> >> >>>> --- Get your facts first, then you can distort them as
you
>> please.--
>> >>
>> >
>> >
>>
>
>
>
> --
> --- Get your facts first, then you can distort them as you please.--
>



-- 
--- Get your facts first, then you can distort them as you please.--

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message