hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Geovanie Marquez <geovanie.marq...@gmail.com>
Subject Re: Why hadoop/hbase uses DNS/hosts/hostname in such a strange way?
Date Thu, 12 Dec 2013 16:00:57 GMT
This may not answer why it is designed this way, but it should give you
more insight into how it is done.

Here is how the network
> happens and the complication may be improved but this is what it is today

On Mon, Dec 9, 2013 at 8:31 PM, Rural Hunter <ruralhunter@gmail.com> wrote:

> Hi,
> I have configured a hadoop/hbase cluster recently and found it's really a
> mess with all those DNS, hostname and /etc/hosts configuration. There are
> many questions related to this all over the internet. So I'm wondering why
> hadoop/hbase designed in such a strange way, which is very abnormal
> comparing with other network/distribution applications. In normal
> applications, DNS is used to indentify other servers(logical or physical),
> not the server itself. But I'm seeing this weired behavior in hadoop/hbase.
> Say we have server1 and server2 configured this way:
> server1(ip
> hostname: server1
> /etc/hosts:
>    localhost,server1
>    server2
> server2(ip
> hostname: server2
> /etc/hosts:
>    localhost,server2
>    server1
> With the configuration above, I'm seesing many cases hadoop/hbase trying
> to connect to localhost while it actaully should connect to another server.
> I believe this is because server1 reported its hostname as 'localhost' to
> server2 and server2 tries to use 'localhost' to connect to server1. But it
> shouldn't work that way. In normal network applications, server2 shouldn't
> try to connect to server1 with what server1 reported. If server2 inits the
> connection, it should use the DNS or /etc/hosts to resolve server1. If
> server 1 inits the connection, server2 should use the ip it gets from the
> already established connection from server1. There shouldn't be any
> confusion or mess.
> I don't see why hadoop/hbase can not use the same logic to handle the
> DNS/hosts/hostname mess. Anyone can resolve my confusion?

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message