cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nux! <>
Subject Re: Libvirt & RBD caching
Date Wed, 18 Feb 2015 23:34:34 GMT
What tests have you done exactly?
I'll try to replicate them in my setup.

FreeBSD also seemed slower to me, though I was more concerned with the network throughput,
it struggles to saturate 1Gbps whereas linux ..just flies.
I would have loved to be able to use pfSense more seriously, but performance sucked in a VM.


Sent from the Delta quadrant using Borg technology!


----- Original Message -----
> From: "Logan Barfield" <>
> To:
> Sent: Wednesday, 18 February, 2015 20:35:12
> Subject: Re: Libvirt & RBD caching

> We're running FreeBSD with VirtIO.  Disk is VirtIO in the libvirt
> config, and shows up as a 'vtbd' device in /dev.  I'm kind of stumped
> on where the performance discrepancy could be coming from.
> Thank You,
> Logan Barfield
> Tranquil Hosting
> On Wed, Feb 18, 2015 at 3:23 PM, Wido den Hollander <> wrote:
>> On 18-02-15 21:00, Logan Barfield wrote:
>>> Our current deployment is KVM with Ceph RBD primary storage.  We have
>>> rbd_cache enabled, and use "cache=none" in Qemu by default.
>>> I've been running some tests to try to figure out why our write speeds
>>> with FreeBSD are significantly lower than Linux.  I was testing both
>>> RBD and local SSD storage, with various cache configurations.  Out of
>>> all of them the only one that performed close to our standard Linux
>>> images was local SSD, Qemu cache=writeback, FreeBSD gpt journal
>>> enabled.
>> I think the main problem is that FreeBSD uses ide/scsi and Linux uses
>> VirtIO.
>> Have you tried running FreeBSD 10 with the OS type set to Ubuntu for
>> example?
>>> I've been reading on various lists the reasons and risks for
>>> cache=none vs cache=writeback:
>>> - cache=none: Safer for live migration
>>> - cache=writeback: Ceph RBD docs claim that this is required for data
>>> integrity when using rbd_cache
>>> From what I can tell performance is generally the same with both,
>>> except in the case of FreeBSD.
>>> What is the current line of thinking on this?  Should be using 'none'
>>> or 'writeback' with RBD by default?  Is 'writeback' considered safe
>>> for live migration?
>> None should still be ok, since librbd does the caching. It's not Qemu
>> who does the caching.
> > Wido

View raw message