hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bill zhu <b...@sitesell.com>
Subject Re: DNS problem
Date Tue, 12 Jan 2010 15:55:25 GMT
Hi, Edward,

Thanks for reply, that is a good point. We didn't have reverse DNS 
setup. If you explain why

possibly reverse lookups for 10.X are going
upstream?

As I understand, hadoop should not be.
Bill

Edward Capriolo wrote:
> On Tue, Jan 12, 2010 at 9:52 AM, bill zhu <bzhu@sitesell.com> wrote:
>   
>> Are the requests for things outside the local domain and/or hosts that are
>> not in the grid?
>> --Sorry, I don't understand this question.
>>
>> Does a trace on the requests verify they are coming from the hadoop grid?
>> --Yes, that is for sure.
>>
>> Are the jobs themselves doing dns lookups of things outside the grid?
>> --No, everything must through master dns server and going outside.
>>
>> Is resolv.conf configured to only point to the local host cache server with
>> the cache then configured to point to the data center cache?
>> --Only localhost in it.
>>
>>
>> Are the TTLs realistic?
>> --Yes, we extended it to 90 days.
>>
>>
>>
>> Allen Wittenauer wrote:
>>     
>>> On 1/11/10 1:43 PM, "bill zhu" <bzhu@sitesell.com> wrote:
>>>
>>>       
>>>> This what we did:
>>>>
>>>> first layer, every nodes have local dns cache.
>>>> second layer, every nodes pointed to master dns cache server.
>>>> Then going outside.
>>>>
>>>>         
>>> That sounds good.
>>>
>>>
>>>       
>>>> After monitoring the traffic from master dns cache server, still lots of
>>>> dns requests going out.
>>>>
>>>>         
>>> Are the requests for things outside the local domain and/or hosts that are
>>> not in the grid?
>>>
>>> Does a trace on the requests verify they are coming from the hadoop grid?
>>>
>>> Are the jobs themselves doing dns lookups of things outside the grid?
>>>
>>> Is resolv.conf configured to only point to the local host cache server
>>> with
>>> the cache then configured to point to the data center cache?
>>>
>>> Are the TTLs realistic?
>>>
>>>
>>>
>>>
>>>       
>>     
>
> Could it be that you do not have your reverse DNS setup for the IP's
> on your cluster and possibly reverse lookups for 10.X are going
> upstream?
>
> Ed
>
>
>   


Mime
View raw message