cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dag Sonstebo (JIRA)" <j...@apache.org>
Subject [jira] [Created] (CLOUDSTACK-9017) VPC VR DHCP broken for multihomed guest VMs
Date Mon, 02 Nov 2015 11:26:27 GMT
Dag Sonstebo created CLOUDSTACK-9017:
----------------------------------------

             Summary: VPC VR DHCP broken for multihomed guest VMs
                 Key: CLOUDSTACK-9017
                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9017
             Project: CloudStack
          Issue Type: Bug
      Security Level: Public (Anyone can view this level - this is the default.)
          Components: SystemVM, Virtual Router
    Affects Versions: 4.5.2
         Environment: CloudStack 4.5.2, XenServer back end.
            Reporter: Dag Sonstebo
            Priority: Minor


Bug: VPC VR DHCP broken for multihomed guest VMs
Affected version: CloudStack 4.5.2 only tested
Summary: When attaching a guest VM to more than one VPC tier DHCP will only work for the last
NIC to be added. This is according to end user new behaviour after the CS4.5.2 upgrade.

Workarounds:
1) Only use single NICs on VPC connected VMs and configure L3 routing and ACLs to handle traffic
between tiers.
2) Configure additional tier NICs with the static IP addresses reported by CloudStack.
		
================================================================================================================

Steps to recreate:
1) Create a VPC with two tiers, in this case
	- VPC on 10.3.0.0/16
	- Tier 1 on 10.3.1.0/24
	- Tier 2 on 10.3.2.0/24

2) Create a new VM attached to tier 1 only. This will cause a new entry to be written to /etc/dhcphosts.txt
on the VPC VR:

root@r-20-VM:~# cat /etc/dhcphosts.txt 
02:00:21:fd:00:08,set:10_3_1_162,10.3.1.162,BatVM2,infinite
root@r-20-VM:~#

When the VM starts up the following is displayed in /var/log/dnsmasq.log when the VM requests
it's IP address:

Oct 30 15:50:12 dnsmasq[8246]: read /etc/hosts - 7 addresses
Oct 30 15:50:12 dnsmasq-dhcp[8246]: read /etc/dhcphosts.txt
Oct 30 15:50:12 dnsmasq-dhcp[8246]: read /etc/dhcpopts.txt
Oct 30 15:50:44 dnsmasq-dhcp[8246]: DHCPDISCOVER(eth2) 02:00:21:fd:00:08 
Oct 30 15:50:44 dnsmasq-dhcp[8246]: DHCPOFFER(eth2) 10.3.1.162 02:00:21:fd:00:08 
Oct 30 15:50:44 dnsmasq-dhcp[8246]: DHCPREQUEST(eth2) 10.3.1.162 02:00:21:fd:00:08 
Oct 30 15:50:44 dnsmasq-dhcp[8246]: DHCPACK(eth2) 10.3.1.162 02:00:21:fd:00:08 BatVM2

The following is displayed in the dnsmasq leases file:

root@r-20-VM:~# cat /var/lib/misc/dnsmasq.leases 
0 02:00:21:fd:00:08 10.3.1.162 BatVM2 *

And the following in the cloud DHCP configuration file:

root@r-20-VM:~# cat /etc/dnsmasq.d/cloud.conf 
dhcp-hostsfile=/etc/dhcphosts.txt
dhcp-range=interface:eth3,set:interface-eth3,10.3.2.1,static
dhcp-option=tag:interface-eth3,15,batvpc.net
dhcp-range=interface:eth2,set:interface-eth2,10.3.1.1,static
dhcp-option=tag:interface-eth2,15,batvpc.net
root@r-20-VM:~# 


3) Checking the VM locally IP configuration will show DHCP lease in place for eth0.

4) Add a new NIC to the VM, attached to Tier 2.	This results in the following entries in the
dnsmasq log:

Oct 30 16:23:02 dnsmasq[8246]: read /etc/hosts - 7 addresses
Oct 30 16:23:02 dnsmasq-dhcp[8246]: read /etc/dhcphosts.txt
Oct 30 16:23:02 dnsmasq-dhcp[8246]: read /etc/dhcpopts.txt
Oct 30 16:23:02 dnsmasq-dhcp[8246]: not giving name BatVM2.batvpc.net to the DHCP lease of
10.3.1.162 because the name exists in /etc/hosts with address 10.3.2.111
Oct 30 16:23:02 dnsmasq-dhcp[8246]: not giving name BatVM2 to the DHCP lease of 10.3.1.162
because the name exists in /etc/hosts with address 10.3.2.111

In other words the Tier 2 address has taken precedence over the initial Tier 1 address.

The /etc/dhcphosts.txt file has now lost the Tier 1 entry and now contains:

root@r-20-VM:~# cat /etc/dhcphosts.txt 
02:00:26:94:00:06,set:10_3_2_111,10.3.2.111,BatVM2,infinite

5) When restarting the VM it will fail to get a DHCP lease on eth0.

Note: in some cases it will reuse the old lease which is cached in the local leases database
- note this IP lease does not come from the VPC VR.

The dnsmasq log will now display the following:

Oct 30 16:30:36 dnsmasq-dhcp[8246]: DHCPREQUEST(eth2) 10.3.1.162 02:00:21:fd:00:08 
Oct 30 16:30:36 dnsmasq-dhcp[8246]: DHCPNAK(eth2) 10.3.1.162 02:00:21:fd:00:08 address not
available
Oct 30 16:30:44 dnsmasq-dhcp[8246]: DHCPDISCOVER(eth2) 02:00:21:fd:00:08 no address available
Oct 30 16:30:58 dnsmasq-dhcp[8246]: DHCPDISCOVER(eth2) 02:00:21:fd:00:08 no address available
Oct 30 16:31:13 dnsmasq-dhcp[8246]: DHCPDISCOVER(eth2) 02:00:21:fd:00:08 no address available
Oct 30 16:31:22 dnsmasq-dhcp[8246]: DHCPDISCOVER(eth2) 02:00:21:fd:00:08 no address available
Oct 30 16:31:32 dnsmasq-dhcp[8246]: DHCPDISCOVER(eth2) 02:00:21:fd:00:08 no address available
Oct 30 16:31:37 dnsmasq-dhcp[8246]: DHCPDISCOVER(eth3) 02:00:26:94:00:06 
Oct 30 16:31:37 dnsmasq-dhcp[8246]: DHCPOFFER(eth3) 10.3.2.111 02:00:26:94:00:06 
Oct 30 16:31:37 dnsmasq-dhcp[8246]: DHCPREQUEST(eth3) 10.3.2.111 02:00:26:94:00:06 
Oct 30 16:31:37 dnsmasq-dhcp[8246]: DHCPACK(eth3) 10.3.2.111 02:00:26:94:00:06 BatVM2

I.e. the VM is not receiving a DHCP offer on eth0 as there are no addresses configured, however
eth1 successfully handshakes.

6) Note - restart of the VPC VR / restart of network with cleanup does not seem to fix the
issue.

7) Just removing the last added NIC does not fix the issue:

The DHCP host file still contains the following, i.e. the host entry from the last added NIC:

root@r-20-VM:~# cat /etc/dhcphosts.txt 
02:00:26:94:00:06,set:10_3_2_111,10.3.2.111,BatVM2,infinite
root@r-20-VM:~# 

Restarting the VM after removal will show:

Oct 30 16:42:00 dnsmasq-dhcp[8246]: DHCPREQUEST(eth2) 10.3.1.162 02:00:21:fd:00:08 
Oct 30 16:42:00 dnsmasq-dhcp[8246]: DHCPNAK(eth2) 10.3.1.162 02:00:21:fd:00:08 address not
available
Oct 30 16:42:08 dnsmasq-dhcp[8246]: DHCPDISCOVER(eth2) 02:00:21:fd:00:08 no address available
Oct 30 16:42:19 dnsmasq-dhcp[8246]: DHCPDISCOVER(eth2) 02:00:21:fd:00:08 no address available
Oct 30 16:42:30 dnsmasq-dhcp[8246]: DHCPDISCOVER(eth2) 02:00:21:fd:00:08 no address available
Oct 30 16:42:50 dnsmasq-dhcp[8246]: DHCPDISCOVER(eth2) 02:00:21:fd:00:08 no address available

I.e. still no DHCP lease on Tier 1.

8) Getting DHCP to work again on the guest VM eth0 involves juggling NICs:
	- Making the last added NIC (eth1) primary.
	- Remove the first NIC (eth0) as discussed in step 7 above. 
	- Readding a new NIC on Tier1.
	- At this point DHCP will work on the Tier 1 NIC, but will be broken on the Tier 2 NIC.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message