cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcus Sorensen (JIRA)" <>
Subject [jira] [Commented] (CLOUDSTACK-4967) vxlan doesn't scale
Date Mon, 28 Oct 2013 23:02:31 GMT


Marcus Sorensen commented on CLOUDSTACK-4967:

Good points. The physical nic name won't be known, however, until we get all the way to implementing
the network via the agent. That might be a bad place to find out that everything is configured
in an unsupported fashion.

We could:
1) limit the max VNI range, documenting such. That is a bit distasteful to me, but would probably
work for the majority of installations.

2) Require the physical nic name as the traffic label, rather than the bridge. This always
struck me as unnecessary, to set up a bridge on the interface that hosts guests, to register
that bridge name, and then have the agent go figure out which physical nic hosts that bridge
so we can make guest bridges on it. All of my systems have unused bridges set up just for
this purpose.  Now, with vxlan, it's even more silly because there are layer 3 interfaces
that don't even work on bridges, but do host vxlan just fine (ipoib, etc).  I will probably
implement something in the pif code that will allow the kvm traffic label to be a physical
device (fallback if no such bridge exists). Vxlan could play off of that by requiring that
it be a physical device rather than a bridge, which would allow you to check the length of
the interface name during zone setup.

3) Just document the limitations and let the end user deal with it.

> vxlan doesn't scale
> -------------------
>                 Key: CLOUDSTACK-4967
>                 URL:
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: KVM
>    Affects Versions: 4.3.0
>            Reporter: Marcus Sorensen
>            Assignee: Yoshikazu Nojima
>             Fix For: 4.3.0
> Failed to create vnet 987529:     inet
brd scope global cloudbr0Error: an inet prefix is expected rather than "239.15.3857.137".Error
> It looks like the vxlan implementation doesn't scale correctly with vxlan's capabilities.
The VNI is supposed to be up to 24 bits (16777216), it should be possible to use high VNI
numbers. The script '' seems to do this:
> local mcastGrp="239.$(( $vxlanId >> 16 % 256 )).$(( $vxlanId >> 8 % 256 )).$((
$vxlanId % 256 ))"
> $vlanid >> 8 %256 (and similar) may need to be ($vxlanId >> 8) % 256
> On a less important note, I should point out that the bridge naming convention will break
in certain rare situations. The max size of a bridge device name is 15 characters. For bond
devices, a VNI above 10 million will not fit, e.g. "brbond0-16000000", or ethernet devices
above 10 "breth10-16000000".  However, these may be quite rare, and changing the naming convention
as we found in 4.2 is a bit painful if it can't be done in a backward compatible way. My first
thought was to have vxlan and vxlan only use hex for it's VNI, that might be ok since it's
never been released yet.

This message was sent by Atlassian JIRA

View raw message