hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Greg Bledsoe <g...@personal.com>
Subject Re: Why Hadoop force using DNS?
Date Mon, 29 Jul 2013 15:33:52 GMT
But even if you have permission to change /etc/hosts, /etc/hosts resolution seems to introduce
instability for the reverse lookup leading to unpredictable results.  Dns gets used and if
this doesn't match your /etc/hosts file, you have problems.  Or am I missing something?


From: Chris Embree <cembree@gmail.com<mailto:cembree@gmail.com>>
Reply-To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>,
"chris@embree.us<mailto:chris@embree.us>" <chris@embree.us<mailto:chris@embree.us>>
Date: Mon, 29 Jul 2013 09:45:22 -0500
To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Subject: Re: Why Hadoop force using DNS?

Just for clarity,  DNS as a service is NOT Required.  Name resolution is.  I use /etc/hosts
files to identify all nodes in my clusters.

One of the reasons for using Names over IP's is ease of use.  I would much rather use a hostname
in my XML to identify NN, JT, etc. vs. some random string of numbers.

On Mon, Jul 29, 2013 at 10:40 AM, Greg Bledsoe <greg@personal.com<mailto:greg@personal.com>>
I can third this concern.  What purpose does this complexity increasing requirement serve?
 Why not remove it?

Greg Bledsoe

From: 武泽胜 <wuzesheng@xiaomi.com<mailto:wuzesheng@xiaomi.com>>
Reply-To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Date: Mon, 29 Jul 2013 08:21:51 -0500
To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Subject: Re: Why Hadoop force using DNS?

I have the same confusion, anyone who can reply to this will be very appreciated.

From: Elazar Leibovich <elazarl@gmail.com<mailto:elazarl@gmail.com>>
Reply-To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Date: Thursday, July 25, 2013 3:51 AM
To: user <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Subject: Why Hadoop force using DNS?

Looking at Hadoop source you can see that Hadoop relies on the fact each node has resolvable

For example, Hadoop 2 namenode reverse look the up of each node that connects to it. Also,
there's no way way to tell a database to advertise an UP as it's address. Setting datanode.network.interface
to, say, eth1, would cause Hadoop to reverse lookup UPs on eth1 and advertise the result.

Why is that? Using plain IPs is simple to set up, and I can't see a reason not to support

View raw message