hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Martyniak <j...@avum.com>
Subject Re: Multiple NIC Cards
Date Tue, 09 Jun 2009 18:35:38 GMT
I haven't even looked into that at all.

I am just trying to get a simple 2 node cluster working with 2 Nics.


On Jun 9, 2009, at 1:41 PM, Edward Capriolo wrote:

> Also if you are using a topology rack map, make sure you scripts
> responds correctly to every possible hostname or IP address as well.
> On Tue, Jun 9, 2009 at 1:19 PM, John Martyniak<john@avum.com> wrote:
>> It seems that this is the issue, as there several posts related to  
>> same
>> topic but with no resolution.
>> I guess the thing of it is that it shouldn't use the hostname of  
>> the machine
>> at all.  If I tell it the master is x and it has an IP Address of  
>> x.x.x.102
>> that should be good enough.
>> And if that isn't the case then I should be able to specify which  
>> network
>> adaptor to use as the ip address that it is going to lookup  
>> against, whether
>> it is by DNS or by /etc/hosts.
>> Because I suspect the problem is that I have named the machine as
>> duey.xxxx.com but have told hadoop that machine is called duey- 
>> direct.
>> Is there work around in 0.19.1?  I am using this with Nutch so  
>> don't have an
>> option to upgrade at this time.
>> -John
>> On Jun 9, 2009, at 11:59 AM, Steve Loughran wrote:
>>> John Martyniak wrote:
>>>> When I run either of those on either of the two machines, it is  
>>>> trying to
>>>> resolve against the DNS servers configured for the external  
>>>> addresses for
>>>> the box.
>>>> Here is the result
>>>> Server:        xxx.xxx.xxx.69
>>>> Address:    xxx.xxx.xxx.69#53
>>> OK. in an ideal world, each NIC has a different hostname. Now, that
>>> confuses code that assumes a host has exactly one hostname, not  
>>> zero or two,
>>> and I'm not sure how well Hadoop handles the 2+ situation (I know  
>>> it doesn't
>>> like 0, but hey, its a distributed application). With separate  
>>> hostnames,
>>> you set hadoop up to work on the inner addresses, and give out the  
>>> inner
>>> hostnames of the jobtracker and namenode. As a result, all traffic  
>>> to the
>>> master nodes should be routed on the internal network

View raw message