cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Hair <j...@greenqloud.com>
Subject Re: Very slow Virtual Router provisioning with 4.9.2.0
Date Wed, 03 May 2017 10:49:00 GMT
Hi Remi,

Do you have a link to the PR that was reverted? And also possibly the code
that makes the password updating more efficient?

Jeff

On Wed, May 3, 2017 at 10:36 AM, Remi Bergsma <RBergsma@schubergphilis.com>
wrote:

> Hi Wido,
>
> When we had similar issues last year, we found that for example comparing
> the iptables rules one-by-one is 1000x slower than simply loading them all
> at once. Boris rewrote this part in our Cosmic fork, may be worth looking
> into this again. The PR to CloudStack was merged, but reverted later, can't
> remember why. We run it in production ever since. Also feeding passwords to
> the passwd server is very inefficient (it operates like a snowball and gets
> slower once you have more VMs). That we also fixed in Cosmic, not sure if
> that patch made it upstream. Wrote it about a year ago already.
>
> We tested applying 10K iptables rules in just a couple of seconds. 1000
> VMs takes a few minutes to deploy.
>
> Generally speaking I'd suggest looking at the logs to find what takes long
> or is executed a lot of times. Iptables and passwd are two to look at.
>
> If you want I can lookup the patches. Not handy on my phone now ;-)
>
> Regards, Remi
> ________________________________
> From: Wido den Hollander <wido@widodh.nl>
> Sent: Tuesday, May 2, 2017 7:57:08 PM
> To: dev@cloudstack.apache.org
> Subject: Very slow Virtual Router provisioning with 4.9.2.0
>
> Hi,
>
> Last night I upgraded a CloudStack 4.5.2 setup to 4.9.2.0. All went well,
> but the VR provisioning is terribly slow which causes all kinds of problems.
>
> The vr_cfg.sh and update_config.py scripts start to run. Restart dnsmasq,
> add metadata, etc.
>
> But for just 1800 hosts this can take up to 2 hours and that causes
> timeouts in the management server and other problems.
>
> 2 hours is just very, very slow. So I am starting to wonder if something
> is wrong here.
>
> Did anybody else see this?
>
> Running Basic Networking with CloudStack 4.9.2.0
>
> Wido
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message