Mailing-List: contact cloudstack-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: cloudstack-dev@incubator.apache.org
Received-SPF: pass (nike.apache.org: domain of mailing.son@gmail.com
 designates 209.85.215.47 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAA3XVTyf9epUK3DBA7Yo_F8kXU8A8Wr_-r2mz2CbteXvthWfrQ@mail.gmail.com>
References: <20121204105828.GD2721@bla.fasel.org>
	<CAA3XVTyf9epUK3DBA7Yo_F8kXU8A8Wr_-r2mz2CbteXvthWfrQ@mail.gmail.com>
Date: Wed, 5 Dec 2012 10:04:54 +0900
Message-ID: 
 <CAMGFgrWfzNcHvcjqM19aiTE0kj4WgG3N2emVi49PEktHCHssdg@mail.gmail.com>
Subject: Re: Router VM peaks at ~920 Mbit/s instad of 4000 Mbit/s network
 offering rate limit
From: Andrew Son <mailing.son@gmail.com>
To: cloudstack-dev@incubator.apache.org
Content-Type: multipart/alternative; boundary=f46d040713e301a0fb04d0109245

--f46d040713e301a0fb04d0109245
Content-Type: text/plain; charset=ISO-8859-1

The network throughput mostly depends on your hardware configuration,
instead of cloudstack.

The key components are
- Network card : intel ixgbe or broadcom ?
- Network device driver version
- Network device driver configuration : GRO
- using VLAN or not :
- using linux bridge or not

If you have default configuration with Xenserver 5.6 sp2, the maximum
network throughput will be about 2 Gbps.

Our experiment shows that
-  instance to instance : 7Gbps
-  instance - router vm - instance : 4Gbps

with
- XenServer 6.0.2
- Custom Intel ixgbe driver
- GRO settings
- ovs (open vswitch)

Cheers,
Choonho Son
2012/12/5 Bryan Whitehead <driver@megahappy.net>

> What is network.throttling.rate and vm.network.throttling.rate set to in
> global settings?
>
> Please correct me if I'm wrong about how these global settings work, but I
> think these values override the templates.
>
>
> On Tue, Dec 4, 2012 at 2:58 AM, Wolfram Schlich
> <lists@wolfram.schlich.org>wrote:
>
> > Hi!
> >
> > Does a router VM have another kind of network rate
> > limitation besides the one from the network offering?
> >
> > We have the network offering set to 4000 Mbit/s
> > which is resembled by XenServer through limiting
> > the interface bandwidth to 500MByte/s (checked that).
> >
> > With unrouted traffic within a CloudStack network
> > (between instances running on different nodes that
> > all have 10 GbE links), the instances peak at the
> > configured 4000 Mbit/s (so, as expected). When
> > doing routed traffic between different CloudStack
> > networks (passing 2 router VMs), the same test peaks
> > at around ~920 Mbit/s while the router VM CPU shows
> > to idle about 99% (25k interrupts/s).
> >
> > The instances used for testing as well as the router
> > VMs run on Citrix XenServer 6.0.2 and are all using
> > xen_netfront as the virtual ethernet driver (which
> > does not even supply a speed value to the VM).
> >
> > I've also looked for any kind of tc/iptables rules
> > on the router VM but couldn't find any that seem
> > relevant for this issue.
> >
> > Ideas, anyone? :-)
> >
> > Cheers,
> > Wolfram
> >
>

--f46d040713e301a0fb04d0109245--