hadoop-zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Hunt <ph...@apache.org>
Subject Re: configuring Zookeeper in HBase with IP addresses only
Date Mon, 24 Aug 2009 22:53:50 GMT
Nice!

Jean-Daniel Cryans wrote:
> Added here http://wiki.apache.org/hadoop/Hbase/Troubleshooting#12
> 
> J-D
> 
> On Mon, Aug 24, 2009 at 5:20 PM, Patrick Hunt<phunt@apache.org> wrote:
>> No worries. The details are actually interesting/useful, you might consider
>> adding to your docs in case another user runs into this.
>>
>> Patrick
>>
>>
>> Jean-Daniel Cryans wrote:
>>> Patrick,
>>>
>>> Basically, yes. Sorry for the lengthy answer ;)
>>>
>>> J-D
>>>
>>> On Mon, Aug 24, 2009 at 5:09 PM, Patrick Hunt<phunt@apache.org> wrote:
>>>> I see, so an inconsistency then wrt name lookup.
>>>>
>>>> Thanks!
>>>>
>>>> Patrick
>>>>
>>>> Jean-Daniel Cryans wrote:
>>>>> Well the situation is that HBase now generates the myid files and to
>>>>> find the id we look in the hbase.zookeeper.quorum configuration that
>>>>> itself generates a temporary zoo.cfg file. To do that we have to
>>>>> somehow match the machine's own knowledge of its address with what's
>>>>> in that list. To find our address we use org.apache.hadoop.net.DNS
>>>>> with the method getDefaultHost and then we go through the list of
>>>>> machines defined in the HBase configuration. What comes out of DNS
>>>>> relies on how the OS is configured or it asks a specified dns server
>>>>> (if provided).
>>>>>
>>>>> So, in David's situation, he specified an IP address and DNS returns
a
>>>>> hostname so we don't get a match. The resolution in that case is to
>>>>> fix the configuration by passing hostnames, to change the OS
>>>>> configuration, to setup a DNS server or to configure/start zookeeper
>>>>> by hand. From what I've seen, that stuff is never easier but eh, we
>>>>> still get you a quorum running in the end :P
>>>>>
>>>>> J-D
>>>>>
>>>>> On Mon, Aug 24, 2009 at 4:37 PM, Patrick Hunt<phunt@apache.org>
wrote:
>>>>>> Hi Jean-Daniel, not sure I get your response fully. Are you saying
that
>>>>>> the
>>>>>> configured ip addr was resolved to a hostname, but that hostname
didn't
>>>>>> match the list of ip addresses used when defining the zk quorum
>>>>>> machines?
>>>>>> Is
>>>>>> there a workaround you could suggest for ppl who don't have DNS
>>>>>> available?
>>>>>> Should an Hbase JIRA be created for this -- ie is it something you
>>>>>> consider
>>>>>> should be fixed/improved?
>>>>>>
>>>>>> Patrick
>>>>>>
>>>>>> Jean-Daniel Cryans wrote:
>>>>>>> Oh ok well HBase relies on the DNS class shipped with Hadoop
to
>>>>>>> determine your address. It will try to use a hostname if possible
but
>>>>>>> what comes out of there really depends on your OS configuration.
In
>>>>>>> your case, that means that it resolved a hostname instead of
an IP
>>>>>>> (which is rare) so you should use it instead.
>>>>>>>
>>>>>>> Also this is HBase-specific, ZK isn't really involved.
>>>>>>>
>>>>>>> J-D
>>>>>>>
>>>>>>> On Mon, Aug 24, 2009 at 3:47 PM, Pythonner<pythonner@gmail.com>
wrote:
>>>>>>>> I forgot to post that line:
>>>>>>>>  <property>
>>>>>>>>  <name>hbase.zookeeper.quorum</name>
>>>>>>>>  <value>192.168.1.xx</value>
>>>>>>>>  </property>
>>>>>>>>
>>>>>>>> ok, I'll check the guide shipped with HBase.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Aug 24, 2009 at 3:43 PM, Jean-Daniel Cryans
>>>>>>>> <jdcryans@apache.org>wrote:
>>>>>>>>
>>>>>>>>> David,
>>>>>>>>>
>>>>>>>>> hbase.master is deprecated in HBase 0.20, instead you
have to
>>>>>>>>> specify
>>>>>>>>> hbase.zookeeper.quorum if you want to use HBase in a
distributed
>>>>>>>>> mode
>>>>>>>>> with a ZK quorum. Please see the Getting Started documentation
>>>>>>>>> shipped
>>>>>>>>> with HBase.
>>>>>>>>>
>>>>>>>>> J-D
>>>>>>>>>
>>>>>>>>> On Mon, Aug 24, 2009 at 3:39 PM, Pythonner<pythonner@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>> Hello,
>>>>>>>>>> this is a follow-up of discussion started on twitter
with
>>>>>>>>>> http://twitter.com/phunt.
>>>>>>>>>>
>>>>>>>>>> I installed HBase 0.20.0 RC2 on Ubuntu server boxes.
>>>>>>>>>>
>>>>>>>>>> If I'm using machines IP in config files (see below),
I get the
>>>>>>>>>> following
>>>>>>>>>> error message:
>>>>>>>>>>
>>>>>>>>>> 'Could not find my address: xyz in list of ZooKeeper
quorum
>>>>>>>>>> servers'
>>>>>>>>> message
>>>>>>>>>> (where 'yxz' is a hostname)
>>>>>>>>>>
>>>>>>>>>> my config is:
>>>>>>>>>>
>>>>>>>>>> hbase-env.sh:
>>>>>>>>>>
>>>>>>>>>> export HBASE_MANAGES_ZK=true
>>>>>>>>>>
>>>>>>>>>> hbase-site.xml:
>>>>>>>>>>
>>>>>>>>>> <configuration>
>>>>>>>>>>
>>>>>>>>>>  <property>
>>>>>>>>>>  <name>hbase.rootdir</name>
>>>>>>>>>>  <value>hdfs://192.168.1.xx:9200/hbase</value>
>>>>>>>>>>  </property>
>>>>>>>>>>
>>>>>>>>>>  <property>
>>>>>>>>>>  <name>hbase.master</name>
>>>>>>>>>>  <value>192.168.1.xx:60000</value>
>>>>>>>>>>  </property>
>>>>>>>>>>
>>>>>>>>>>  <property>
>>>>>>>>>>  <name>hbase.cluster.distributed</name>
>>>>>>>>>>  <value>true</value>
>>>>>>>>>>  </property>
>>>>>>>>>>
>>>>>>>>>> </configuration>
>>>>>>>>>>
>>>>>>>>>> from vanilla Ubuntu server install, I removed the
127.0.1.1 line
>>>>>>>>>> from
>>>>>>>>>> /etc/hosts
>>>>>>>>>>
>>>>>>>>>> Is it supposed to work well with IP addresses only?
>>>>>>>>>>
>>>>>>>>>> David
>>>>>>>>>>
>>>>>>>> --
>>>>>>>> Balie - Baseline Information Extraction
>>>>>>>> http://balie.sourceforge.net
>>>>>>>> [Open Source ~ 100% Java ~ Using Weka ~ Multilingual]
>>>>>>>>

Mime
View raw message