cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Trevor Francis <>
Subject Advanced Networking
Date Tue, 16 Oct 2012 20:00:41 GMT
So, with the help of the fine gents on the IRC channel (thank you KDamage), I have successfully
managed to crash my network over 20 times due to ARP floods coming from the bridges that I
have created. I attempted to create a highly available network by utilizing bonded bridges
split amongst trunked switch pairs and designing physical networks to take advantage of those

Here is what my network looks like

Host (6 total ethernet ports)

Eth0, Eth1 = Bond0, cloudVirBr10 (bridge) . Bonding mode 6, STP enabled.
Eth2, Eth3 = Bond2, cloudVirBr12 (bridge) . Bonding mode 6, STP enabled.
Eth4, Eth5 = Bond1, cloudVirBr11 (bridge) . Bonding mode 6, STP enabled.

I have the bonds split amongst trunked switches. 

Switch1A - eth0 , Switch1B, eth1 trunked between each other, STP enabled.
Switch2A - eth4, Switch2B, eth5 trunked between each other, STP enabled.
Switch3A - eth2, Switch3B, eth3 trunked between each other, STP enabled.

The switch pairs are totally isolated from each other (ie, no interswitch trunking).

So, you can turn off any switch or pull any cable and the network still works fine.

Switch1A and Switch1B have internet delivered to them through HSRP (2 drops), so if either
of the switches crashes, the internet continues.

I have been working for over a month to figure out why the floods occur, but we just cant
seem to figure it out.

The hosts are running Cent 6.3 and KVM HVs.
Our goal was to have Management and Storage traffic go over Bond2, Guest traffic go over Bond1,
and Public Internet traffic go over Bond0. We created physical networks in advanced networking
with those settings and KVM tags corresponding to our bridge names. 
Our application requirement require that we have 2 ethernet connections assigned directly
to the linux VMs (eth0 = public internet, eth1 = guest network). We cannot use NAT between
a public and a private network.
To achieve this, we created a standard Isolated guest network and then created a shared network
that assigned a pool of public IPs directly to the machines.
We can successfully turn a VM up with a public IP without storms and turn a VM up with a guest
IP without storms. But when we turn a VM up with both, the storms start immediately, killing
the public network.

We are considering abandoing KVM in lieu of Xenserver, but we were told that there was no
way to successfully separate our public traffic to go over a bonded bride and have our guest
traffic go over a different bridge.

My brain still works in "Old Networking" mode, so the new cloud networking stuff is throwing
me for a loop. 

Seeing what our network consists of, do you recommend doing things a different way? We are
considering swapping Xen for the KVM HVs and are looking at getting maximum performance while
still maintaining full 2N redundancy.


View raw message