hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bill zhu <b...@sitesell.com>
Subject Re: DNS problem
Date Mon, 11 Jan 2010 21:43:30 GMT
This what we did:

first layer, every nodes have local dns cache.
second layer, every nodes pointed to master dns cache server.
Then going outside.

After monitoring the traffic from master dns cache server, still lots of 
dns requests going out.

Bill
Allen Wittenauer wrote:
> Sorry, by local caching server, I meant, on each node, not one for the data
> center.
>
>
> On 1/11/10 1:28 PM, "bill zhu" <bzhu@sitesell.com> wrote:
>
>   
>> Hi, Allen,
>>
>> Thanks for your feed back. I did user unbound as our local cache DNS
>> server, but still lots unreason DNS requests going outside, we have a
>> VPN between hadoop group machines and our main data center, these DNS
>> traffic did block the VPN. We don't know if there is a way to control
>> this DNS storm and let it run slowly?
>>
>> Bill
>>
>> Allen Wittenauer wrote:
>>     
>>> Either use nscd or local caching dns server (bind, djbdns, whatever).  Some
>>> will argue for just pushing the info to /etc/hosts, but I wouldn't. :)
>>>
>>>
>>> On 1/11/10 1:22 PM, "bill zhu" <bzhu@sitesell.com> wrote:
>>>
>>>   
>>>       
>>>> Dear, all,
>>>>
>>>> We setup 23 nodes and one master server in a cluster, but we suffered
>>>> some problem of DNS. At the begin of Fetch, we found out hadoop generate
>>>> so many DNS requests like a storm. If anyway we can control this kind of
>>>> DNS requests and let them run slowly?
>>>>
>>>> Any reply will be appreciated.
>>>>
>>>> Bill
>>>>     
>>>>         
>>>
>>>   
>>>       
>
>
>
>   


Mime
View raw message