cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chiradeep Vittal <Chiradeep.Vit...@citrix.com>
Subject Re: Virtual Router: DHCP and 2-second DNS outages
Date Tue, 30 Apr 2013 23:52:40 GMT


On 4/30/13 3:26 PM, "Dennis Lawler" <dlawler@gmail.com> wrote:

>Every time a new VM is started up, there is a 2 second outage in DNS
>services that can cause problems in guest VMs that use the router VM for
>DNS.
>
> 
>
>For Cloudstack configurations using both DHCP and DNS services on the
>router
>VM (both implemented with dnsmasq), there is currently a 2 second DNS
>service outage every time a new VM is instantiated
>
> 
>
>The source of this outage is in edithosts.sh, which uses "service dnsmasq
>restart" to pick up the freshly added DNS and DHCP entries.
>
>Restarting the dnsmasq service triggers a sleep for 2 seconds after
>killing
>dnsmasq before starting it back up again.
>
> 
>
>An obvious solution would be to replace "service dnsmasq restart" with
>"kill
>-s 1 $pid" (SIGHUP) so that dnsmasq reads the new DHCP entries without
>restarting, as in dnsmasq_edithosts.sh (external dhcp).
> 
>
>Unfortunately, this solution is flawed because dnsmasq SIGHUP handling
>does
>not expire in-memory DHCP leases in dnsmasq and all leases are infinite by
>default.

Aha! That's why SIGHUP didn't work consistently. This has been bugging me
for a long time.

>Thus, this will only work if the guest VM performs a DHCP release on
>shutdown, which cannot always be guaranteed.
>
> 
>
>A few possible solutions off the top of my head:
>
>1.       Separate DNS and DHCP services.  While DHCP services still
>experience an outage during VM,  DNS will not necessarily be impacted if
>implemented correctly.
>
>2.       Use SIGHUP with dnsmasq and implement a removeDhcpEntry interface
>for network appliances to force a DHCP release whenever a NIC / IP is
>deallocated.  This can use dhcp_release to simulate a DHCP release on the
>router VM.
>Catch: dhcp_release is not available for Debian 6.0.  The System VM needs
>to
>be updated to at least Debian 7.0, or the dnsmasq-tools .deb from 7.0
>would
>need to be included in the System VM image.

There is going to be a new system vm based on 7.0 for the upcoming
release. This should work with earlier releases as well.
https://cwiki.apache.org/confluence/x/UlHVAQ

>
>3.       Change DHCP to have a shorter lease, track de-allocation of IPs
>separately from VM destruction.
>Catch: This may cause occasional IP pool exhaustion depending on
>allocation
>of the guest IP range and the rate of VM destruction / instantiation in
>the
>network.
>
> 
>
>Thoughts?
>


Mime
View raw message