incubator-cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcus Sorensen <shadow...@gmail.com>
Subject Re: VM router spawning multiple public nics
Date Tue, 28 Aug 2012 20:39:36 GMT
Yes, that looks like it would work for me, however that's not
something that would ever make it into master, right? Essentially
killing tagging for the public, private, and guest traffic labels?
There's also still the issue of not being able to differentiate
between multiple untagged networks, if we wanted to add an IP to a
router it might not know which untagged interface to apply it to.

On Tue, Aug 28, 2012 at 2:32 PM, Edison Su <Edison.su@citrix.com> wrote:
>
>
>> -----Original Message-----
>> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>> Sent: Tuesday, August 28, 2012 12:23 PM
>> To: cloudstack-dev@incubator.apache.org
>> Subject: Re: VM router spawning multiple public nics
>>
>> Thanks for pointing me in the right direction. I've reviewed this code
>> in a bit more detail, and it seems like it's accomplishing the
>> following:
>>
>> 1. get all network interfaces currently connected to the running VM
>> (a.k.a vnet devices via libvirt)
>> 2. find out which vlans these network interfaces are bridged to, store
>> this in a hash map of vlan ids and nics
>> 3. get all ip addresses to be added to the VM
>> 4. for each ip, get the configured vlan id for the ip, compare it to
>> the hash map of existing vlan ids and nics
>> 5. if the required vlan id is not found in the hash map, create a new
>> nic
>> 6. assign the ip to the nic identified by the vlan id key in the hash
>> map
>>
>> In this case, we're getting a vlan id returned in step 2 for a bridged
>> nic whose network is defined as untagged in the cloudstack db,
>> therefore in step 5 we never match as already having a nic for
>> 'untagged'. I wrote a big long response discussing this issue, but as
>> I began to dig further I realized that aside from my particular case,
>> untagged vlans in general are just broken (for example they can't be
>> dealt with uniquely in the current IpAssocCommand code, given the hash
>> map) and it would require more effort than I have time for now to make
>> things work. If the code were already in place to differentiate
>> between multiple untagged nics I think that fixing my problem would be
>> trivial, but since its not, I'll just find an alternative solution.
>>
>
> The untagged network usually means "untagged", no vlan on the bridge...
> In your case, the untagged network actually has vlan(tagged) on the bridge, thus getting
things confused.
> Will this patch(http://pastebin.com/HJXzZwKp) work for you?
>
>> On Mon, Aug 27, 2012 at 10:46 PM, Marcus Sorensen <shadowsor@gmail.com>
>> wrote:
>> > ...
>> >             Integer nicPos = 0;
>> >             for (InterfaceDef nic : nics) {
>> >                 if
>> (nic.getBrName().equalsIgnoreCase(_linkLocalBridgeName)) {
>> >                     vlanAllocatedToVM.put("LinkLocal", nicPos);
>> >                 } else {
>> >                     String vlanId =
>> getVlanIdFromBridge(nic.getBrName());
>> >                     if (vlanId != null) {
>> >                         vlanAllocatedToVM.put(vlanId, nicPos);
>> >                     } else {
>> >                         vlanAllocatedToVM.put(Vlan.UNTAGGED, nicPos);
>> >                     }
>> >                 }
>> >                 nicPos++;
>> >             }
>> >             IpAddressTO[] ips = cmd.getIpAddresses();
>> >             int i = 0;
>> >             String result = null;
>> >             int nicNum = 0;
>> >             for (IpAddressTO ip : ips) {
>> >                 if (!vlanAllocatedToVM.containsKey(ip.getVlanId())) {
>> >                     /* plug a vif into router */
>> >                     VifHotPlug(conn, routerName, ip.getVlanId(),
>> >                             ip.getVifMacAddress());
>> >                     vlanAllocatedToVM.put(ip.getVlanId(), nicPos++);
>> >                 }
>> > ...
>> >
>> > Looks like the getVlanIdFromBridge might be a bit misleading. I am
>> > running my guest public traffic on a 'cloudbr470', which is a bridge
>> > to eth2.470, yet I configured this network as 'untagged' because I
>> > have a vlan 470 available on eth3 for cloudstack to autoassign (eth3
>> > is where all of my stuff will be autoassigned). So I'm not 100% sure
>> > yet what's going on here but it seems as though the above is not
>> > setting any 'Vlan.UNTAGGED', since it finds a vlan number for
>> > eth2.470, but when it enumerates the IPs for the router, it then runs
>> > ip.getVlanId() and doesn't find a nic for the untagged IP and creates
>> > one.
>> >
>> >
>> > I realize this is perhaps an uncommon case, but a bug nonetheless.
>> > I'll play with the code a bit and see if I can come up with a
>> > solution. I'm thinking I can look at the nic's broadcast URI and see
>> > if it's supposed to be untagged, then add to vlanAllocatedToVM
>> > appropriately, off the top of my head something like:
>> >
>> >                     String vlanId =
>> getVlanIdFromBridge(nic.getBrName());
>> >                     if (vlanId != null &&
>> > !nic.getBroadcastUri().toString().contains("untagged") {
>> >                         vlanAllocatedToVM.put(vlanId, nicPos);
>> >                     } else {
>> >                         vlanAllocatedToVM.put(Vlan.UNTAGGED, nicPos);
>> >                     }
>> >
>> >
>> >
>> > On Mon, Aug 27, 2012 at 6:42 PM, Edison Su <Edison.su@citrix.com>
>> wrote:
>> >> Possible bug in in kvm code: LibvirtComputingResource-
>> >execute(IpAssocCommand cmd)-> VifHotPlug, which is only place adding
>> nic into router vm.
>> >> Turn on agent log, then take a look what happened.
>> >>
>> >>> -----Original Message-----
>> >>> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>> >>> Sent: Monday, August 27, 2012 5:10 PM
>> >>> To: cloudstack-dev@incubator.apache.org
>> >>> Subject: VM router spawning multiple public nics
>> >>>
>> >>> I've got two zones running the same build of cloudstack (a recent
>> copy
>> >>> of master). One of them creates routers that turn into ugly
>> >>> multi-headed beasts, and by that I mean that any time I create a
>> port
>> >>> forwarding or iptables rule for that router I get a new public NIC
>> >>> with an identical IP address, I have an instance with a few tens of
>> >>> NICs.  My guess is that some script isn't detecting that there's
>> >>> already a NIC with the public IP on it.  It looks fine in the
>> >>> database, there is only one public NIC defined in the nics table.
>> >>> I'll troubleshoot it tomorrow, but if anyone knows where I should
>> >>> begin the headstart would be appreciated.
>> >>>
>> >>> Thanks

Mime
View raw message