cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrija Panic <andrija.pa...@gmail.com>
Subject Re: Caching modes
Date Tue, 20 Feb 2018 15:41:06 GMT
I advise (or not...depends on point of view) to stay that way - because
when you activate write-back cache - live migrations will stop, and this
makes *Enable maintenance mode (put host into maintenance)* impossible.

I would perhaps suggest that there is documentation for "advanced users" or
similar, that will say "it's possible to enable this and this way via DB
hack, but be warned on live migration consequences etc..." since this will
render more problems if people start using it.

If you choose to do, let me know, I can write that (documentation) briefly.

Not to mention it can be unsafe (power failure - less possible I guess, but
rare kernel panic etc might have it's consequences I assume)

It does indeed increase performance on NFS much, but not necessarily on
CEPH (if you are using lirbd cache on client side as explained above)

On 20 February 2018 at 15:48, Rafael Weingärtner <
rafaelweingartner@gmail.com> wrote:

> Yes. Weird enough, the code is using the value in the database if it is
> provided there, but there is no easy way for users to change that
> configuration in the database. ¯\_(ツ)_/¯
>
> On Tue, Feb 20, 2018 at 11:45 AM, Andrija Panic <andrija.panic@gmail.com>
> wrote:
>
> > So it seems that just passing the cachemode value to API is not there, or
> > somehow messedup, but deployVM process does read DB values from
> > disk_offering table for sure, and applies it to XML file for KVM.
> > This is above ACS 4.8.x.
> >
> >
> > On 20 February 2018 at 15:44, Andrija Panic <andrija.panic@gmail.com>
> > wrote:
> >
> > > I have edited the disk_offering table, in the cache_mode just enter
> > > "writeback". Stop and start VM, and it will pickup/inherit the
> cache_mode
> > > from it's parrent offering
> > > This also applies to Compute/Service offering, again inside
> disk_offering
> > > table - just tested both
> > >
> > > i.e.
> > >
> > > UPDATE `cloud`.`disk_offering` SET `cache_mode`='writeback' WHERE
> > > `id`=102; # Compute Offering (Service offering)
> > > UPDATE `cloud`.`disk_offering` SET `cache_mode`='writeback' WHERE
> > > `id`=114; #data disk offering
> > >
> > > Before SQL:
> > >
> > > root@ix1-c7-4:~# virsh dumpxml i-2-10-VM | grep cache -A2
> > >       <driver name='qemu' type='qcow2' cache='none'/>
> > >       <source file='/mnt/63a3ae7b-9ea9-3884-
> a772-1ea939ef6ec3/1b655159-
> > > ae10-41cf-8987-f1cfb47fe453'/>
> > >       <target dev='vda' bus='virtio'/>
> > > --
> > >       <driver name='qemu' type='qcow2' cache='none'/>
> > >       <source file='/mnt/63a3ae7b-9ea9-3884-
> a772-1ea939ef6ec3/09bdadcb-
> > > ec6e-4dda-b37b-17b1a749257f'/>
> > >       <target dev='vdb' bus='virtio'/>
> > > --
> > >
> > > STOP and START VM = after SQL
> > >
> > > root@ix1-c7-4:~# virsh dumpxml i-2-10-VM | grep cache -A2
> > >       <driver name='qemu' type='qcow2' cache='writeback'/>
> > >       <source file='/mnt/63a3ae7b-9ea9-3884-
> a772-1ea939ef6ec3/1b655159-
> > > ae10-41cf-8987-f1cfb47fe453'/>
> > >       <target dev='vda' bus='virtio'/>
> > > --
> > >       <driver name='qemu' type='qcow2' cache='writeback'/>
> > >       <source file='/mnt/63a3ae7b-9ea9-3884-
> a772-1ea939ef6ec3/09bdadcb-
> > > ec6e-4dda-b37b-17b1a749257f'/>
> > >       <target dev='vdb' bus='virtio'/>
> > > --
> > >
> > >
> > >
> > > On 20 February 2018 at 14:03, Rafael Weingärtner <
> > > rafaelweingartner@gmail.com> wrote:
> > >
> > >> I have no idea how it can change the performance. If you look at the
> > >> content of the commit you provided, it is only the commit that enabled
> > the
> > >> use of getCacheMode from disk offerings. However, it is not exposing
> any
> > >> way to users to change that value/configuration in the database. I
> might
> > >> have missed it; do you see any API methods that receive the parameter
> > >> "cacheMode" and then pass this parameter to a "diskOffering" object,
> and
> > >> then persist/update this object in the database?
> > >>
> > >> May I ask how are you guys changing the cacheMode configuration?
> > >>
> > >> On Tue, Feb 20, 2018 at 9:56 AM, Paul Angus <paul.angus@shapeblue.com
> >
> > >> wrote:
> > >>
> > >> > I'm working with some guys who are experimenting with the setting
as
> > if
> > >> > definitely seems to change the performance of data disks.  It also
> > >> changes
> > >> > the XML of the VM which is created.
> > >> >
> > >> > p.s.
> > >> > I've found this commit;
> > >> >
> > >> > https://github.com/apache/cloudstack/commit/1edaa36cc68e845a
> > >> 42339d5f267d49
> > >> > c82343aefb
> > >> >
> > >> > so I've got something to investigate now, but API documentation must
> > >> > definitely be askew.
> > >> >
> > >> >
> > >> >
> > >> > paul.angus@shapeblue.com
> > >> > www.shapeblue.com
> > >> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > >> > @shapeblue
> > >> >
> > >> >
> > >> >
> > >> >
> > >> > -----Original Message-----
> > >> > From: Rafael Weingärtner [mailto:rafaelweingartner@gmail.com]
> > >> > Sent: 20 February 2018 12:31
> > >> > To: dev <dev@cloudstack.apache.org>
> > >> > Subject: Re: Caching modes
> > >> >
> > >> > This cache mode parameter does not exist in "CreateDiskOfferingCmd"
> > >> > command. I also checked some commits from 2, 3, 4 and 5 years ago,
> and
> > >> > this parameter was never there. If you check the API in [1], you can
> > see
> > >> > that it is not an expected parameter. Moreover, I do not see any use
> > of
> > >> > "setCacheMode" in the code (in case it is updated by some other
> > method).
> > >> > Interestingly enough, the code uses "getCacheMode".
> > >> >
> > >> > In summary, it is not a feature, and it does not work. It looks like
> > >> some
> > >> > leftover from dark ages when people could commit anything and then
> > they
> > >> > would just leave a half implementation there in our code base.
> > >> >
> > >> > [1]
> > >> > https://cloudstack.apache.org/api/apidocs-4.11/apis/
> > >> > createDiskOffering.html
> > >> >
> > >> >
> > >> > On Tue, Feb 20, 2018 at 9:19 AM, Andrija Panic <
> > andrija.panic@gmail.com
> > >> >
> > >> > wrote:
> > >> >
> > >> > > I can also assume that "cachemode" as API parameter is not
> > supported,
> > >> > > since when creating data disk offering via GUI also doesn't set
it
> > in
> > >> > DB/table.
> > >> > >
> > >> > > CM:    create diskoffering name=xxx displaytext=xxx
> > storagetype=shared
> > >> > > disksize=1024 cachemode=writeback
> > >> > >
> > >> > > this also does not set cachemode in table... my guess it's not
> > >> > > implemented in API
> > >> > >
> > >> > > Let me know if I can help with any testing here.
> > >> > >
> > >> > > Cheers
> > >> > >
> > >> > > On 20 February 2018 at 13:09, Andrija Panic <
> > andrija.panic@gmail.com>
> > >> > > wrote:
> > >> > >
> > >> > > > Hi Paul,
> > >> > > >
> > >> > > > not helping directly answering your question, but here are
some
> > >> > > > observations and "warning" if client's are using write-back
> cache
> > on
> > >> > > > KVM level
> > >> > > >
> > >> > > >
> > >> > > > I have (long time ago) tested performance in 3 combinations
> (this
> > >> > > > was not really thorough testing but a brief testing with
FIO and
> > >> > > > random IO WRITE)
> > >> > > >
> > >> > > > - just CEPH rbd cache (on KVM side)
> > >> > > >            i.e. [client]
> > >> > > >                  rbd cache = true
> > >> > > >                  rbd cache writethrough until flush = true
> > >> > > >                  #(this is default 32MB per volume, afaik
> > >> > > >
> > >> > > > - just KMV write-back cache (had to manually edit disk_offering
> > >> > > > table to activate cache mode, since when creating new disk
> > offering
> > >> > > > via GUI, the disk_offering tables was NOT populated with
> > >> > > > "write-back" sertting/value
> > >> > > ! )
> > >> > > >
> > >> > > > - both CEPH and KVM write-back cahce active
> > >> > > >
> > >> > > > My observations were like following, but would be good to
> actually
> > >> > > confirm
> > >> > > > by someone else:
> > >> > > >
> > >> > > > - same performance with only CEPH caching or with only KVM
> caching
> > >> > > > - a bit worse performance with both CEPH and KVM caching
active
> > >> > > > (nonsense combination, I know...)
> > >> > > >
> > >> > > >
> > >> > > > Please keep in mind, that some ACS functionality, KVM
> > >> > > > live-migrations on shared storage (NFS/CEPH) are NOT supported
> > when
> > >> > > > you use KVM write-back cache, since this is considered "unsafe"
> > >> > migration, more info here:
> > >> > > > https://doc.opensuse.org/documentation/leap/virtualization/
> > >> > > html/book.virt/
> > >> > > > cha.cachemodes.html#sec.cache.mode.live.migration
> > >> > > >
> > >> > > > or in short:
> > >> > > > "
> > >> > > > The libvirt management layer includes checks for migration
> > >> > > > compatibility based on several factors. If the guest storage
is
> > >> > > > hosted on a clustered file system, is read-only or is marked
> > >> > > > shareable, then the cache mode is ignored when determining
if
> > >> > > > migration can be allowed. Otherwise libvirt will not allow
> > migration
> > >> > > > unless the cache mode is set to none. However, this restriction
> > can
> > >> > > > be overridden with the “unsafe” option to the migration
APIs,
> > which
> > >> > > > is also supported by virsh, as for example in
> > >> > > >
> > >> > > > virsh migrate --live --unsafe
> > >> > > > "
> > >> > > >
> > >> > > > Cheers
> > >> > > > Andrija
> > >> > > >
> > >> > > >
> > >> > > > On 20 February 2018 at 11:24, Paul Angus <
> > paul.angus@shapeblue.com>
> > >> > > wrote:
> > >> > > >
> > >> > > >> Hi Wido,
> > >> > > >>
> > >> > > >> This is for KVM (with Ceph backend as it happens), the
API
> > >> > > >> documentation is out of sync with UI capabilities, so
I'm
> trying
> > to
> > >> > > >> figure out if we
> > >> > > >> *should* be able to set cacheMode for root disks.  It
seems to
> > make
> > >> > > quite a
> > >> > > >> difference to performance.
> > >> > > >>
> > >> > > >>
> > >> > > >>
> > >> > > >> paul.angus@shapeblue.com
> > >> > > >> www.shapeblue.com
> > >> > > >> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
> > >> > > >>
> > >> > > >>
> > >> > > >>
> > >> > > >>
> > >> > > >> -----Original Message-----
> > >> > > >> From: Wido den Hollander [mailto:wido@widodh.nl]
> > >> > > >> Sent: 20 February 2018 09:03
> > >> > > >> To: dev@cloudstack.apache.org
> > >> > > >> Subject: Re: Caching modes
> > >> > > >>
> > >> > > >>
> > >> > > >>
> > >> > > >> On 02/20/2018 09:46 AM, Paul Angus wrote:
> > >> > > >> > Hey guys,
> > >> > > >> >
> > >> > > >> > Can anyone shed any light on write caching in CloudStack?
> > >> >  cacheMode
> > >> > > >> is available through the UI for data disks (but not
root
> disks),
> > >> but
> > >> > not
> > >> > > >> documented as an API option for data or root disks (although
is
> > >> > > documented
> > >> > > >> as a response for data disks).
> > >> > > >> >
> > >> > > >>
> > >> > > >> What hypervisor?
> > >> > > >>
> > >> > > >> In case of KVM it's passed down to XML which then passes
it to
> > >> > Qemu/KVM
> > >> > > >> which then handles the caching.
> > >> > > >>
> > >> > > >> The implementation varies per hypervisor, so that should
be the
> > >> > > question.
> > >> > > >>
> > >> > > >> Wido
> > >> > > >>
> > >> > > >>
> > >> > > >> > #huh?
> > >> > > >> >
> > >> > > >> > thanks
> > >> > > >> >
> > >> > > >> >
> > >> > > >> >
> > >> > > >> > paul.angus@shapeblue.com
> > >> > > >> > www.shapeblue.com
> > >> > > >> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
> > >> > > >> >
> > >> > > >> >
> > >> > > >> >
> > >> > > >>
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > --
> > >> > > >
> > >> > > > Andrija Panić
> > >> > > >
> > >> > >
> > >> > >
> > >> > >
> > >> > > --
> > >> > >
> > >> > > Andrija Panić
> > >> > >
> > >> >
> > >> >
> > >> >
> > >> > --
> > >> > Rafael Weingärtner
> > >> >
> > >>
> > >>
> > >>
> > >> --
> > >> Rafael Weingärtner
> > >>
> > >
> > >
> > >
> > > --
> > >
> > > Andrija Panić
> > >
> >
> >
> >
> > --
> >
> > Andrija Panić
> >
>
>
>
> --
> Rafael Weingärtner
>



-- 

Andrija Panić

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message