hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Edward Capriolo <edlinuxg...@gmail.com>
Subject Re: Multiple NIC Cards
Date Tue, 09 Jun 2009 17:41:07 GMT
Also if you are using a topology rack map, make sure you scripts
responds correctly to every possible hostname or IP address as well.

On Tue, Jun 9, 2009 at 1:19 PM, John Martyniak<john@avum.com> wrote:
> It seems that this is the issue, as there several posts related to same
> topic but with no resolution.
> I guess the thing of it is that it shouldn't use the hostname of the machine
> at all.  If I tell it the master is x and it has an IP Address of x.x.x.102
> that should be good enough.
> And if that isn't the case then I should be able to specify which network
> adaptor to use as the ip address that it is going to lookup against, whether
> it is by DNS or by /etc/hosts.
> Because I suspect the problem is that I have named the machine as
> duey.xxxx.com but have told hadoop that machine is called duey-direct.
> Is there work around in 0.19.1?  I am using this with Nutch so don't have an
> option to upgrade at this time.
> -John
> On Jun 9, 2009, at 11:59 AM, Steve Loughran wrote:
>> John Martyniak wrote:
>>> When I run either of those on either of the two machines, it is trying to
>>> resolve against the DNS servers configured for the external addresses for
>>> the box.
>>> Here is the result
>>> Server:        xxx.xxx.xxx.69
>>> Address:    xxx.xxx.xxx.69#53
>> OK. in an ideal world, each NIC has a different hostname. Now, that
>> confuses code that assumes a host has exactly one hostname, not zero or two,
>> and I'm not sure how well Hadoop handles the 2+ situation (I know it doesn't
>> like 0, but hey, its a distributed application). With separate hostnames,
>> you set hadoop up to work on the inner addresses, and give out the inner
>> hostnames of the jobtracker and namenode. As a result, all traffic to the
>> master nodes should be routed on the internal network

View raw message