cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Remi Bergsma <>
Subject Re: Very slow Virtual Router provisioning with
Date Wed, 03 May 2017 11:28:18 GMT

The patches I talked about:

1) Iptables speed improvement 
Was reverted due to a licensing issue. 

2) Passwd speed improvement 

By now, these are rather old patches so they need some work before they apply to CloudStack

Regards, Remi

On 03/05/2017, 12:49, "Jeff Hair" <> wrote:

    Hi Remi,
    Do you have a link to the PR that was reverted? And also possibly the code
    that makes the password updating more efficient?
    On Wed, May 3, 2017 at 10:36 AM, Remi Bergsma <>
    > Hi Wido,
    > When we had similar issues last year, we found that for example comparing
    > the iptables rules one-by-one is 1000x slower than simply loading them all
    > at once. Boris rewrote this part in our Cosmic fork, may be worth looking
    > into this again. The PR to CloudStack was merged, but reverted later, can't
    > remember why. We run it in production ever since. Also feeding passwords to
    > the passwd server is very inefficient (it operates like a snowball and gets
    > slower once you have more VMs). That we also fixed in Cosmic, not sure if
    > that patch made it upstream. Wrote it about a year ago already.
    > We tested applying 10K iptables rules in just a couple of seconds. 1000
    > VMs takes a few minutes to deploy.
    > Generally speaking I'd suggest looking at the logs to find what takes long
    > or is executed a lot of times. Iptables and passwd are two to look at.
    > If you want I can lookup the patches. Not handy on my phone now ;-)
    > Regards, Remi
    > ________________________________
    > From: Wido den Hollander <>
    > Sent: Tuesday, May 2, 2017 7:57:08 PM
    > To:
    > Subject: Very slow Virtual Router provisioning with
    > Hi,
    > Last night I upgraded a CloudStack 4.5.2 setup to All went well,
    > but the VR provisioning is terribly slow which causes all kinds of problems.
    > The and scripts start to run. Restart dnsmasq,
    > add metadata, etc.
    > But for just 1800 hosts this can take up to 2 hours and that causes
    > timeouts in the management server and other problems.
    > 2 hours is just very, very slow. So I am starting to wonder if something
    > is wrong here.
    > Did anybody else see this?
    > Running Basic Networking with CloudStack
    > Wido

View raw message