hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: Way to Specify HBase master?
Date Thu, 26 Nov 2009 00:20:06 GMT
This would be a good FAQ entry Andrew, your explanations are way
better than mine. The master publishing itself is new in 0.20 and I
guess others will hit this issue with Ubuntu.

J-D

On Tue, Nov 24, 2009 at 2:03 PM, Andrew Purtell <apurtell@apache.org> wrote:
> Good to hear Mark.
>
> HBase is sensitive to this in a way that Hadoop is not because HBase region servers and
clients do not use a static configuration to locate the master. If you check your Hadoop config,
I suspect you'll find you configured the namenode location using either a hostname that did
not resolve to a localhost address, or a non-localhost IP address. Likewise perhaps the jobtracker
location. So datanodes, tasktrackers, clients, etc. would use the static configuration no
matter what IP address the namenode, jobtracker, etc. was self-discovering. However the HBase
master publishes its location to clients via Zookeeper. This is done to support multimaster
operation (failover). So if the HBase master self-discovers its location as a localhost address,
then it will publish that. Region servers or clients which go to Zookeeper for the master
location will get back an address in that case only useful if they happen to be co-located
with the master.
>
>   - Andy
>
>
>
> ________________________________
> From: Mark Vigeant <mark.vigeant@riskmetrics.com>
> To: "hbase-user@hadoop.apache.org" <hbase-user@hadoop.apache.org>
> Sent: Tue, November 24, 2009 1:41:15 PM
> Subject: RE: Way to Specify HBase master?
>
> Ah, thank you so much Andy!
>
> You know, I had just gotten hadoop working again (it turns out it too was running with
master 127.0.1.1 which is odd because it worked while hbase didn't...) by changing the hostnames
on my machines and doing a lot of frustrated restarting but finally everything worked.
>
> And I just deleted the 127/8 things in my /etc/hosts and it looks like hbase is FINALLY
running! Woo!
>
> -Mark
>
> -----Original Message-----
> From: Andrew Purtell [mailto:apurtell@apache.org]
> Sent: Tuesday, November 24, 2009 4:28 PM
> To: hbase-user@hadoop.apache.org
> Subject: Re: Way to Specify HBase master?
>
> If you have an entry in your /etc/hosts file on the master that causes it to resolve
its hostname to a localhost address (anything in 127/8) , then this will happen:
> 1) The master will publish its address into zookeeper as a localhost address.
> 2) The region servers will attempt to contact the master on the localhost address...
which is their localhost of course
> 3) The connection fails unless the region server and the master happen to be running
on the same node.
>
> In your case, the solution is to remove all entries in /etc/hosts which cause resolution
of the local host name to a localhost address and substitute them with either a functional
DNS setup or a /etc/hosts file which resolves your host names to the IP addresses assigned
on the shared LAN segment (i.e. eth0 interfaces).
>
>   - Andy
>
>
>
>
> ________________________________
> From: Mark Vigeant <mark.vigeant@riskmetrics.com>
> To: "hbase-user@hadoop.apache.org" <hbase-user@hadoop.apache.org>
> Sent: Tue, November 24, 2009 9:21:01 AM
> Subject: RE: Way to Specify HBase master?
>
> Wait, I think there's been some confusion: the master resolves itself to 127.0.1.1, not
127.0.0.1
>
> I've looked at the HMaster logs (http://pastebin.com/m4ecf071e) and it says that in fact
what the region servers are reading is true: the master is running as "hadoop 127.0.1.1."
A few interesting things to note are: the ROOT region is not set, and the /hbase/rs node is
completely empty.
>
> Is this still the matter of a host-name issue? I notice there are dns properties for
all aspects of hbase, should I tamper with them?
>
> -----Original Message-----
> From: jdcryans@gmail.com [mailto:jdcryans@gmail.com] On Behalf Of Jean-Daniel Cryans
> Sent: Tuesday, November 24, 2009 11:40 AM
> To: hbase-user@hadoop.apache.org
> Subject: Re: Way to Specify HBase master?
>
> It seems to be the same problems isn't it? The master resolves itself
> to 127.0.0.1 and all region servers, when communicating with
> Zookeeper, read the master's znode with the local address but since
> there isn't a master there, it can't connect.
>
> The reason why you can't specify a master is because HBase is
> dynamically multi-master e.g. you can start 100 masters (one is
> elected), kill some backups, get new ones, etc. So each node, master
> and region server, in HBase has to be able to publish its own address.
>
> Like Stack said, playing with your hosts file should be enough, just
> make sure that the hadoop config is still sane (and try to see why it
> fails if so).
>
> If you have a DNS server, you can also set the following values:
>
>  <property>
>    <name>hbase.zookeeper.dns.interface</name>
>    <value>default</value>
>    <description>The name of the Network Interface from which a ZooKeeper server
>      should report its IP address.
>    </description>
>  </property>
>  <property>
>    <name>hbase.zookeeper.dns.nameserver</name>
>    <value>default</value>
>    <description>The host name or IP address of the name server (DNS)
>      which a ZooKeeper server should use to determine the host name used by the
>      master for communication and display purposes.
>    </description>
>  </property>
>
> J-D
>
> On Tue, Nov 24, 2009 at 8:28 AM, Mark Vigeant
> <mark.vigeant@riskmetrics.com> wrote:
>> So I tried to change the hostname and it messed up my hadoop cluster... it appears
that everything else runs fine on the host hadoop 127.0.1.1
>>
>> I'll change strategies here: my regionservers see the host and try to connect to
it, but the connection is refused. Does anyone have any ideas on how I can change that?
>>
>> Here's a snippet of the log from my regionserver: http://pastebin.com/m6bbcb432
>> -----Original Message-----
>> From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of stack
>> Sent: Monday, November 23, 2009 5:28 PM
>> To: hbase-user@hadoop.apache.org
>> Subject: Re: Way to Specify HBase master?
>>
>> This sounds like basic networking issue where host lookup is finding
>> 127.0.0.1.   Can you get someone to help you setup your networking on your
>> machine.  Else play w/ your /etc/hosts.  Seems like your host name resolves
>> to 127.0.0.1.
>> St.Ack
>>
>> On Mon, Nov 23, 2009 at 12:31 PM, Mark Vigeant <mark.vigeant@riskmetrics.com
>>> wrote:
>>
>>> I'm trying to play around with HBase but it hasn't been working for me.
>>> From the look of the logs, it appears that the master is running on
>>> 127.0.1.1 instead of my machine (fry) and so all of the regionservers start,
>>> but the logs say they cannot connect to the master. From the master logs, it
>>> thinks there are 0 regionservers and thus won't do anything for me. I'm
>>> using HBase 20.2. How can I set the IP address of the master machine?
>>>
>>> Mark Vigeant
>>> RiskMetrics Group, Inc.
>
>
>
>
> This email message and any attachments are for the sole use of the intended recipients
and may contain proprietary and/or confidential information which may be privileged or otherwise
protected from disclosure. Any unauthorized review, use, disclosure or distribution is prohibited.
If you are not an intended recipient, please contact the sender by reply email and destroy
the original message and any copies of the message as well as any attachments to the original
message.
>
>
>
>

Mime
View raw message