hadoop-zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: configuring Zookeeper in HBase with IP addresses only
Date Mon, 24 Aug 2009 21:16:43 GMT
Patrick,

Basically, yes. Sorry for the lengthy answer ;)

J-D

On Mon, Aug 24, 2009 at 5:09 PM, Patrick Hunt<phunt@apache.org> wrote:
> I see, so an inconsistency then wrt name lookup.
>
> Thanks!
>
> Patrick
>
> Jean-Daniel Cryans wrote:
>>
>> Well the situation is that HBase now generates the myid files and to
>> find the id we look in the hbase.zookeeper.quorum configuration that
>> itself generates a temporary zoo.cfg file. To do that we have to
>> somehow match the machine's own knowledge of its address with what's
>> in that list. To find our address we use org.apache.hadoop.net.DNS
>> with the method getDefaultHost and then we go through the list of
>> machines defined in the HBase configuration. What comes out of DNS
>> relies on how the OS is configured or it asks a specified dns server
>> (if provided).
>>
>> So, in David's situation, he specified an IP address and DNS returns a
>> hostname so we don't get a match. The resolution in that case is to
>> fix the configuration by passing hostnames, to change the OS
>> configuration, to setup a DNS server or to configure/start zookeeper
>> by hand. From what I've seen, that stuff is never easier but eh, we
>> still get you a quorum running in the end :P
>>
>> J-D
>>
>> On Mon, Aug 24, 2009 at 4:37 PM, Patrick Hunt<phunt@apache.org> wrote:
>>>
>>> Hi Jean-Daniel, not sure I get your response fully. Are you saying that
>>> the
>>> configured ip addr was resolved to a hostname, but that hostname didn't
>>> match the list of ip addresses used when defining the zk quorum machines?
>>> Is
>>> there a workaround you could suggest for ppl who don't have DNS
>>> available?
>>> Should an Hbase JIRA be created for this -- ie is it something you
>>> consider
>>> should be fixed/improved?
>>>
>>> Patrick
>>>
>>> Jean-Daniel Cryans wrote:
>>>>
>>>> Oh ok well HBase relies on the DNS class shipped with Hadoop to
>>>> determine your address. It will try to use a hostname if possible but
>>>> what comes out of there really depends on your OS configuration. In
>>>> your case, that means that it resolved a hostname instead of an IP
>>>> (which is rare) so you should use it instead.
>>>>
>>>> Also this is HBase-specific, ZK isn't really involved.
>>>>
>>>> J-D
>>>>
>>>> On Mon, Aug 24, 2009 at 3:47 PM, Pythonner<pythonner@gmail.com> wrote:
>>>>>
>>>>> I forgot to post that line:
>>>>>  <property>
>>>>>  <name>hbase.zookeeper.quorum</name>
>>>>>  <value>192.168.1.xx</value>
>>>>>  </property>
>>>>>
>>>>> ok, I'll check the guide shipped with HBase.
>>>>>
>>>>>
>>>>> On Mon, Aug 24, 2009 at 3:43 PM, Jean-Daniel Cryans
>>>>> <jdcryans@apache.org>wrote:
>>>>>
>>>>>> David,
>>>>>>
>>>>>> hbase.master is deprecated in HBase 0.20, instead you have to specify
>>>>>> hbase.zookeeper.quorum if you want to use HBase in a distributed
mode
>>>>>> with a ZK quorum. Please see the Getting Started documentation shipped
>>>>>> with HBase.
>>>>>>
>>>>>> J-D
>>>>>>
>>>>>> On Mon, Aug 24, 2009 at 3:39 PM, Pythonner<pythonner@gmail.com>
wrote:
>>>>>>>
>>>>>>> Hello,
>>>>>>> this is a follow-up of discussion started on twitter with
>>>>>>> http://twitter.com/phunt.
>>>>>>>
>>>>>>> I installed HBase 0.20.0 RC2 on Ubuntu server boxes.
>>>>>>>
>>>>>>> If I'm using machines IP in config files (see below), I get the
>>>>>>> following
>>>>>>> error message:
>>>>>>>
>>>>>>> 'Could not find my address: xyz in list of ZooKeeper quorum servers'
>>>>>>
>>>>>> message
>>>>>>>
>>>>>>> (where 'yxz' is a hostname)
>>>>>>>
>>>>>>> my config is:
>>>>>>>
>>>>>>> hbase-env.sh:
>>>>>>>
>>>>>>> export HBASE_MANAGES_ZK=true
>>>>>>>
>>>>>>> hbase-site.xml:
>>>>>>>
>>>>>>> <configuration>
>>>>>>>
>>>>>>>  <property>
>>>>>>>  <name>hbase.rootdir</name>
>>>>>>>  <value>hdfs://192.168.1.xx:9200/hbase</value>
>>>>>>>  </property>
>>>>>>>
>>>>>>>  <property>
>>>>>>>  <name>hbase.master</name>
>>>>>>>  <value>192.168.1.xx:60000</value>
>>>>>>>  </property>
>>>>>>>
>>>>>>>  <property>
>>>>>>>  <name>hbase.cluster.distributed</name>
>>>>>>>  <value>true</value>
>>>>>>>  </property>
>>>>>>>
>>>>>>> </configuration>
>>>>>>>
>>>>>>> from vanilla Ubuntu server install, I removed the 127.0.1.1 line
from
>>>>>>> /etc/hosts
>>>>>>>
>>>>>>> Is it supposed to work well with IP addresses only?
>>>>>>>
>>>>>>> David
>>>>>>>
>>>>>
>>>>> --
>>>>> Balie - Baseline Information Extraction
>>>>> http://balie.sourceforge.net
>>>>> [Open Source ~ 100% Java ~ Using Weka ~ Multilingual]
>>>>>
>

Mime
View raw message