cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wei ZHOU <ustcweiz...@gmail.com>
Subject Re: ACS - Some VMs unable to get DHCP IP from VR
Date Tue, 08 Nov 2016 08:17:47 GMT
Hi,

I think it is related to multiple ip ranges in a shared network.
If I change the ip in /etc/dhcphosts.txt to another ip in other range, I
got the same error log in dnsmasq.log

We do not use multiple vlan or ip ranges in a shared network. I believe few
of us have the same use case.
If possible, it would be better to add the new ip range as a new shared
network.

-Wei


2016-11-07 21:35 GMT+01:00 Cloud List <cloud-list@sg.or.id>:

> Hi Chiradeep and Wei, thanks for your reply.
>
> Wei, here's the result you requested:
>
> root@r-4155-VM:/var/log2# cat /etc/dnsmasq.conf |grep -v "^#" |grep -v
> "^$"
> domain-needed
> bogus-priv
> resolv-file=/etc/dnsmasq-resolv.conf
> local=/cs1cloud.internal/
> except-interface=eth1
> except-interface=eth2
> except-interface=lo
> no-dhcp-interface=eth1
> no-dhcp-interface=eth2
> expand-hosts
> domain=cs1cloud.internal
> domain=cs1cloud.internal
> domain=cs1cloud.internal
> dhcp-range=X.Y.202.1,static
> dhcp-hostsfile=/etc/dhcphosts.txt
> dhcp-ignore=tag:!known
> dhcp-option=15,"cs1cloud.internal"
> dhcp-option=vendor:MSFT,2,1i
> dhcp-boot=pxelinux.0
> enable-tftp
> tftp-root=/opt/tftpboot
> dhcp-lease-max=2100
> domain=cs1cloud.internal
> log-dhcp
> log-facility=/var/log2/dnsmasq.log
> conf-dir=/etc/dnsmasq.d
> dhcp-optsfile=/etc/dhcpopts.txt
> dhcp-option=option:router,X.Y.202.1
> dhcp-option=6,X.Y.202.2,8.8.8.8,8.8.4.4
> dhcp-client-update
>
> We actually have 3 class-C subnets assigned to the network.
>
> X.Y.202.0/24
> X.Y.203.0/24
> Z.A.107.0/24
>
> After enabling log-dhcp as per Chiradeep Vittal, I can see that the
> available DHCP subnet is only the first subnet.
>
> Nov  7 20:27:49 dnsmasq-dhcp[22462]: 1979424125 available DHCP subnet:
> X.Y.202.2/255.255.255.0
> Nov  7 20:27:49 dnsmasq-dhcp[22462]: 1979424125 available DHCP subnet:
> X.Y.202.1/255.255.255.0
> Nov  7 20:27:49 dnsmasq-dhcp[22462]: 1979424125 client provides name:
> Debian-81-64b
> Nov  7 20:27:49 dnsmasq-dhcp[22462]: 1979424125 DHCPDISCOVER(eth0)
> 06:c5:38:01:13:40 ignored
> Nov  7 20:27:51 dnsmasq-dhcp[22462]: 2670909966 available DHCP subnet:
> X.Y.202.2/255.255.255.0
> Nov  7 20:27:51 dnsmasq-dhcp[22462]: 2670909966 available DHCP subnet:
> X.Y.202.1/255.255.255.0
> Nov  7 20:27:51 dnsmasq-dhcp[22462]: 2670909966 client provides name:
> WIN-H4INMOBBRJA
> Nov  7 20:27:51 dnsmasq-dhcp[22462]: 2670909966 vendor class: MSFT 5.0
> Nov  7 20:27:51 dnsmasq-dhcp[22462]: 2670909966 DHCPDISCOVER(eth0)
> 06:31:ac:01:13:f0 ignored
>
> Coincidentally, all affected VMs are coming from the second subnet:
> X.Y.203.0/24, while the third subnet is rarely used.
>
> Do we need to specifically include the second and third subnets into the
> dnsmasq.conf file? I tried adding below line:
>
> dhcp-range=X.Y.203.1,static
>
> so it will become:
>
> dhcp-range=X.Y.202.1,static
> dhcp-range=X.Y.203.1,static
>
> but it doesn't seem to work.
>
> Any advice is highly appreciated.
>
> Thank you.
>
> On Tue, Nov 8, 2016 at 4:19 AM, Wei ZHOU <ustcweizhou@gmail.com> wrote:
>
> > can you paste the result of following command?
> >
> > cat /etc/dnsmasq.conf |grep -v "^#" |grep -v "^$"
> >
> > -Wei
> >
> >
> > 2016-11-07 20:27 GMT+01:00 Cloud List <cloud-list@sg.or.id>:
> >
> > > Hi Wei,
> > >
> > > In addition,
> > >
> > > The VR is serving a shared not isolated network, meaning the IP it
> serves
> > > is 'guest' not 'public' IP. Will that make a difference on the iptables
> > > command we need to execute?
> > >
> > > Looking forward to your reply, thank you.
> > >
> > > Cheers.
> > >
> > >
> > > On Tue, Nov 8, 2016 at 3:19 AM, Cloud List <cloud-list@sg.or.id>
> wrote:
> > >
> > > > Hi Wei and Ozhan,
> > > >
> > > > Thanks for your reply.
> > > >
> > > > The problem doesn't affect only Debian-based guest VMs, but also
> > affected
> > > > some Windows and Ubuntu-based VMs as well. I have executed the
> command
> > on
> > > > the VR and reset the NIC of the guest VM, but unfortunately the issue
> > > still
> > > > persists.
> > > >
> > > > iptables -t mangle -A POSTROUTING -p udp -m udp --dport 68 -j
> CHECKSUM
> > > > --checksum-fill
> > > >
> > > > After issuing the above command on VR and reset the NIC on guest vm
> > > > (ifdown eth0, ifup eth0):
> > > >
> > > > On VR's /var/log/dnsmasq.log:
> > > >
> > > > Nov  7 19:09:22 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0)
> > 06:b1:22:01:13:17
> > > > ignored
> > > > Nov  7 19:09:25 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0)
> > 06:b1:22:01:13:17
> > > > ignored
> > > > Nov  7 19:09:30 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0)
> > 06:b1:22:01:13:17
> > > > ignored
> > > > Nov  7 19:09:36 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0)
> > 06:b1:22:01:13:17
> > > > ignored
> > > >
> > > > On the guest VM:
> > > >
> > > > root@vm:~# ifup eth0
> > > > Internet Systems Consortium DHCP Client 4.2.2
> > > > Copyright 2004-2011 Internet Systems Consortium.
> > > > All rights reserved.
> > > > For info, please visit https://www.isc.org/software/dhcp
> > > >
> > > > Listening on LPF/eth0/06:b1:22:01:13:17
> > > > Sending on LPF/eth0/06:b1:22:01:13:17
> > > > Sending on Socket/fallback
> > > > DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 4
> > > > DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 5
> > > > DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval
> 64DHCPDISCOVER
> > > on
> > > > eth0 to 255.255.255.255 port 67 interval 14
> > > > DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 9
> > > > DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 10
> > > > DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 13
> > > > No DHCOFFERS received.
> > > > No working leases in persistent database - sleeping.
> > > >
> > > > I also tried to execute similar hotfix as mentioned on the bug
> report:
> > > >
> > > > iptables -A POSTROUTING -t mangle -p udp --dport bootpc -j CHECKSUM
> > > > --checksum-fill
> > > >
> > > > The problem still persists. Any further advice on this is highly
> > > > appreciated.
> > > >
> > > > Looking forward to your reply, thank you.
> > > >
> > > > Cheers.
> > > >
> > > >
> > > > On Tue, Nov 8, 2016 at 2:41 AM, Wei ZHOU <ustcweizhou@gmail.com>
> > wrote:
> > > >
> > > >> GOOD point.
> > > >>
> > > >> @CloudList, can you try again after executing the following command
> in
> > > VR
> > > >> ?
> > > >>
> > > >> iptables -t mangle -A POSTROUTING -p udp -m udp --dport 68 -j
> CHECKSUM
> > > >> --checksum-fill
> > > >>
> > > >> -Wei
> > > >>
> > > >> 2016-11-07 14:46 GMT+01:00 Özhan Rüzgar Karaman <
> > > oruzgarkaraman@gmail.com
> > > >> >:
> > > >>
> > > >> > Hi;
> > > >> > Whats your problematic vm's type is it Debian? Maybe you are
> > affected
> > > >> from
> > > >> > the bug CLOUDSTACK-8326? I do not know if this bug has effected
on
> > ACS
> > > >> 4.2
> > > >> > release but i know that it effects release 4.8.x 4.9.x
> > > >> >
> > > >> > Thanks
> > > >> > Özhan
> > > >> >
> > > >> > On Mon, Nov 7, 2016 at 4:36 PM, Cloud List <cloud-list@sg.or.id>
> > > wrote:
> > > >> >
> > > >> > > Hi Wei,
> > > >> > >
> > > >> > > Thanks for your reply.
> > > >> > >
> > > >> > > I checked and I can confirm that the mac address is listed
on
> > > >> > > /etc/dhcphosts.txt in VR.
> > > >> > >
> > > >> > > For example:
> > > >> > >
> > > >> > > Nov  7 13:30:19 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0)
> > > >> 06:31:ac:01:13:YY
> > > >> > > ignored
> > > >> > > Nov  7 13:30:24 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0)
> > > >> 06:31:ac:01:13:YY
> > > >> > > ignored
> > > >> > > Nov  7 13:30:33 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0)
> > > >> 06:31:ac:01:13:YY
> > > >> > > ignored
> > > >> > >
> > > >> > > root@r-4155-VM:/var/log# grep 06:31:ac:01:13:f0
> > /etc/dhcphosts.txt
> > > >> > > 06:31:ac:01:13:YY,X.X.X.X,vm-name,infinite
> > > >> > >
> > > >> > > YY - last two digits of the mac address
> > > >> > > X.X.X.X - ip address which is supposed to be allocated to
the VM
> > > >> > >
> > > >> > > Any advice how can I troubleshoot this further?
> > > >> > >
> > > >> > > Looking forward to your reply, thank you.
> > > >> > >
> > > >> > > Cheers.
> > > >> > >
> > > >> > >
> > > >> > > On Mon, Nov 7, 2016 at 9:22 PM, Wei ZHOU <ustcweizhou@gmail.com
> >
> > > >> wrote:
> > > >> > >
> > > >> > > > If the mac address is not listed in /etc/dhcphosts.txt
in VR,
> > the
> > > >> > request
> > > >> > > > will be ignored.
> > > >> > > >
> > > >> > > > Can you give more details so we can reproduce it and
fix it ?
> > > >> > > >
> > > >> > > > -Wei
> > > >> > > >
> > > >> > > > 2016-11-07 13:44 GMT+01:00 Cloud List <cloud-list@sg.or.id>:
> > > >> > > >
> > > >> > > > > Hi,
> > > >> > > > >
> > > >> > > > > After our upgrade from CloudStack 4.2 to 4.8.1.1,
we noted
> > that
> > > >> some
> > > >> > > (but
> > > >> > > > > not all) of our VMs are not able to get DHCP from
the VR.
> This
> > > >> gives
> > > >> > > > > problem when the VM is restarted and cannot get
up and
> running
> > > >> > because
> > > >> > > > > unable to get IP.
> > > >> > > > >
> > > >> > > > > I logged in to the VR and found below messages
showing that
> > the
> > > >> DHCP
> > > >> > > > server
> > > >> > > > > is ignoring the request from the VM.
> > > >> > > > >
> > > >> > > > > Nov  7 12:19:43 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0)
> > > >> > > 06:44:da:01:13:98
> > > >> > > > > ignored
> > > >> > > > > Nov  7 12:19:52 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0)
> > > >> > > 06:05:64:01:13:d3
> > > >> > > > > ignored
> > > >> > > > > Nov  7 12:19:53 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0)
> > > >> > > 06:44:da:01:13:98
> > > >> > > > > ignored
> > > >> > > > > Nov  7 12:20:04 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0)
> > > >> > > 06:44:da:01:13:98
> > > >> > > > > ignored
> > > >> > > > > Nov  7 12:20:11 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0)
> > > >> > > 06:05:64:01:13:d3
> > > >> > > > > ignored
> > > >> > > > > Nov  7 12:20:22 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0)
> > > >> > > 06:44:da:01:13:98
> > > >> > > > > ignored
> > > >> > > > > Nov  7 12:20:30 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0)
> > > >> > > 06:44:da:01:13:98
> > > >> > > > > ignored
> > > >> > > > > Nov  7 12:20:42 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0)
> > > >> > > 06:44:da:01:13:98
> > > >> > > > > ignored
> > > >> > > > > Nov  7 12:21:02 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0)
> > > >> > > 06:44:da:01:13:98
> > > >> > > > > ignored
> > > >> > > > > Nov  7 12:21:19 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0)
> > > >> > > 06:44:da:01:13:98
> > > >> > > > > ignored
> > > >> > > > >
> > > >> > > > > Any reason why the dnsmasq service ignored the
request from
> > the
> > > VM
> > > >> > and
> > > >> > > > how
> > > >> > > > > can we fix that?
> > > >> > > > >
> > > >> > > > > At the moment, the only workaround we can do is
to configure
> > the
> > > >> IP
> > > >> > > > address
> > > >> > > > > statically to the servelet, which is not practical.
> > > >> > > > >
> > > >> > > > > Any advice is greatly appreciated.
> > > >> > > > >
> > > >> > > > > Thank you.
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message