cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Spencer Harmon <>
Subject VM MTU and Routing Between Hosts
Date Tue, 15 Dec 2015 14:55:23 GMT
Sorry if this is a double post. I missed the subscribe email, so if my initial email made it
to the mailing list, I apologize. 

I'm trying to create a cloudstack setup (4.5.2 with KVM) with a "Guest"
network on two physical interfaces. I have two bridges on top of a single bond, one on the
bond itself (cloudbr0) and one on a vlan subinterface of the bond (cloudbr1). I have interface
tags configured (in this case they match my interface names, cloudbr0 and cloudbr1) on the
guest networks. The reason for this is that I wish to use vxlan for isolation for the majority
of clients, but I have a few clients that will eventually need to have physical hosts on their
cloudstack network (necessitating using vlan for isolation). I also need the management interfaces
to use 1500mtu network interfaces, but I need the vxlan interfaces to use 9000mtu to account
for the overhead of vxlan.

I added a network offering, making sure to put in the tag of cloudbr1's interface. But when
I deploy a network using this network offering, it uses cloudbr0 instead.

It seems like traffic between VMs on separate hosts, e.g. from a VM on one host to a virtual
router on another will use the fqdn of the target host and whatever kernel routing is applicable.
I.e., if I change my hosts file to direct connections from hyper0 to hyper1 on the cloudbr1
IP rather than the cloudbr0 IP, it will route over the intended interface. The caveat with
this approach is that the VM was still created with MTU 1500 and the bridge interface for
the VM at MTU 1450. I'd like for the VM to detect the MTU of the intended interface, have
MTU 9000 with a bridge at MTU 8950. The latter case works as intended if I set both cloudbr0
and cloudbr1 to MTU 9000, but that presents a problem for my management network. 

[root@hyper1 ~]# ip -d link show vxlan766720: vxlan7667:
<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN 
    link/ether ba:42:75:e2:23:6c brd ff:ff:ff:ff:ff:ff    vxlan id 7667
group dev cloudbr0 port 32768 61000 ttl 10 ageing 300 I would post some logs,
but I'm not sure which (if any) are relevant.
I've grepped for "cloudbr0" and "cloudbr1" in all logs, and didn't see an instance of either.
Please advise if I can provide any logs or information that may help.

View raw message