incubator-cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Edison Su <Edison...@citrix.com>
Subject RE: VM router spawning multiple public nics
Date Tue, 28 Aug 2012 20:32:24 GMT


> -----Original Message-----
> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
> Sent: Tuesday, August 28, 2012 12:23 PM
> To: cloudstack-dev@incubator.apache.org
> Subject: Re: VM router spawning multiple public nics
> 
> Thanks for pointing me in the right direction. I've reviewed this code
> in a bit more detail, and it seems like it's accomplishing the
> following:
> 
> 1. get all network interfaces currently connected to the running VM
> (a.k.a vnet devices via libvirt)
> 2. find out which vlans these network interfaces are bridged to, store
> this in a hash map of vlan ids and nics
> 3. get all ip addresses to be added to the VM
> 4. for each ip, get the configured vlan id for the ip, compare it to
> the hash map of existing vlan ids and nics
> 5. if the required vlan id is not found in the hash map, create a new
> nic
> 6. assign the ip to the nic identified by the vlan id key in the hash
> map
> 
> In this case, we're getting a vlan id returned in step 2 for a bridged
> nic whose network is defined as untagged in the cloudstack db,
> therefore in step 5 we never match as already having a nic for
> 'untagged'. I wrote a big long response discussing this issue, but as
> I began to dig further I realized that aside from my particular case,
> untagged vlans in general are just broken (for example they can't be
> dealt with uniquely in the current IpAssocCommand code, given the hash
> map) and it would require more effort than I have time for now to make
> things work. If the code were already in place to differentiate
> between multiple untagged nics I think that fixing my problem would be
> trivial, but since its not, I'll just find an alternative solution.
> 

The untagged network usually means "untagged", no vlan on the bridge...
In your case, the untagged network actually has vlan(tagged) on the bridge, thus getting things
confused.
Will this patch(http://pastebin.com/HJXzZwKp) work for you?

> On Mon, Aug 27, 2012 at 10:46 PM, Marcus Sorensen <shadowsor@gmail.com>
> wrote:
> > ...
> >             Integer nicPos = 0;
> >             for (InterfaceDef nic : nics) {
> >                 if
> (nic.getBrName().equalsIgnoreCase(_linkLocalBridgeName)) {
> >                     vlanAllocatedToVM.put("LinkLocal", nicPos);
> >                 } else {
> >                     String vlanId =
> getVlanIdFromBridge(nic.getBrName());
> >                     if (vlanId != null) {
> >                         vlanAllocatedToVM.put(vlanId, nicPos);
> >                     } else {
> >                         vlanAllocatedToVM.put(Vlan.UNTAGGED, nicPos);
> >                     }
> >                 }
> >                 nicPos++;
> >             }
> >             IpAddressTO[] ips = cmd.getIpAddresses();
> >             int i = 0;
> >             String result = null;
> >             int nicNum = 0;
> >             for (IpAddressTO ip : ips) {
> >                 if (!vlanAllocatedToVM.containsKey(ip.getVlanId())) {
> >                     /* plug a vif into router */
> >                     VifHotPlug(conn, routerName, ip.getVlanId(),
> >                             ip.getVifMacAddress());
> >                     vlanAllocatedToVM.put(ip.getVlanId(), nicPos++);
> >                 }
> > ...
> >
> > Looks like the getVlanIdFromBridge might be a bit misleading. I am
> > running my guest public traffic on a 'cloudbr470', which is a bridge
> > to eth2.470, yet I configured this network as 'untagged' because I
> > have a vlan 470 available on eth3 for cloudstack to autoassign (eth3
> > is where all of my stuff will be autoassigned). So I'm not 100% sure
> > yet what's going on here but it seems as though the above is not
> > setting any 'Vlan.UNTAGGED', since it finds a vlan number for
> > eth2.470, but when it enumerates the IPs for the router, it then runs
> > ip.getVlanId() and doesn't find a nic for the untagged IP and creates
> > one.
> >
> >
> > I realize this is perhaps an uncommon case, but a bug nonetheless.
> > I'll play with the code a bit and see if I can come up with a
> > solution. I'm thinking I can look at the nic's broadcast URI and see
> > if it's supposed to be untagged, then add to vlanAllocatedToVM
> > appropriately, off the top of my head something like:
> >
> >                     String vlanId =
> getVlanIdFromBridge(nic.getBrName());
> >                     if (vlanId != null &&
> > !nic.getBroadcastUri().toString().contains("untagged") {
> >                         vlanAllocatedToVM.put(vlanId, nicPos);
> >                     } else {
> >                         vlanAllocatedToVM.put(Vlan.UNTAGGED, nicPos);
> >                     }
> >
> >
> >
> > On Mon, Aug 27, 2012 at 6:42 PM, Edison Su <Edison.su@citrix.com>
> wrote:
> >> Possible bug in in kvm code: LibvirtComputingResource-
> >execute(IpAssocCommand cmd)-> VifHotPlug, which is only place adding
> nic into router vm.
> >> Turn on agent log, then take a look what happened.
> >>
> >>> -----Original Message-----
> >>> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
> >>> Sent: Monday, August 27, 2012 5:10 PM
> >>> To: cloudstack-dev@incubator.apache.org
> >>> Subject: VM router spawning multiple public nics
> >>>
> >>> I've got two zones running the same build of cloudstack (a recent
> copy
> >>> of master). One of them creates routers that turn into ugly
> >>> multi-headed beasts, and by that I mean that any time I create a
> port
> >>> forwarding or iptables rule for that router I get a new public NIC
> >>> with an identical IP address, I have an instance with a few tens of
> >>> NICs.  My guess is that some script isn't detecting that there's
> >>> already a NIC with the public IP on it.  It looks fine in the
> >>> database, there is only one public NIC defined in the nics table.
> >>> I'll troubleshoot it tomorrow, but if anyone knows where I should
> >>> begin the headstart would be appreciated.
> >>>
> >>> Thanks

Mime
View raw message