cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rafael Weingärtner <rafaelweingart...@gmail.com>
Subject Re: Caching modes
Date Wed, 21 Feb 2018 12:15:22 GMT
You merged from branch andrijapanic-patch-1, into master. However, this
process was done in your own repository and not ACS.
You had this PR opened: https://github.com/apache/cloudstack-docs/pull/22,
and it was merged on Nov 9, 2017.

That is the content you are seeing on link http://docs.cloudstack.apache.
org/en/latest/networking/vxlan.html#important-note-on-
max-number-of-multicast-groups-and-thus-vxlan-intefaces



On Wed, Feb 21, 2018 at 9:08 AM, Andrija Panic <andrija.panic@gmail.com>
wrote:

> Rafael, I just successfully merged (strange?)
> https://github.com/andrijapanic/cloudstack-docs/pull/1 and I can see
> changes are publicly available on
> http://docs.cloudstack.apache.org/en/latest/networking/
> vxlan.html#important-note-on-max-number-of-multicast-
> groups-and-thus-vxlan-intefaces
>
> Is this normal, that I can merge my own pull request on cloudstack-doc repo
> ? There is no limitations (I was able to make PR and merge myself)
>
> On 21 February 2018 at 00:58, Andrija Panic <andrija.panic@gmail.com>
> wrote:
>
> > pls merge also https://github.com/apache/cloudstack-docs-admin/pull/48
> >
> > just correct code block syntax (to display code properly)
> >
> > On 20 February 2018 at 21:02, Rafael Weingärtner <
> > rafaelweingartner@gmail.com> wrote:
> >
> >> Thanks, we will proceed reviweing
> >>
> >> On Tue, Feb 20, 2018 at 3:12 PM, Andrija Panic <andrija.panic@gmail.com
> >
> >> wrote:
> >>
> >> > Here it is:
> >> >
> >> > https://github.com/apache/cloudstack-docs-admin/pull/47
> >> >
> >> > Added KVM online storage migration (atm only CEPH/NFS to SolidFire,
> new
> >> in
> >> > 4.11 release)
> >> > Added KVM cache mode setup and limitations.
> >> >
> >> >
> >> > Cheers
> >> >
> >> > On 20 February 2018 at 16:49, Rafael Weingärtner <
> >> > rafaelweingartner@gmail.com> wrote:
> >> >
> >> > > If you are willing to write it down, please do so, and open a PR.
We
> >> will
> >> > > review and merged it afterwards.
> >> > >
> >> > > On Tue, Feb 20, 2018 at 12:41 PM, Andrija Panic <
> >> andrija.panic@gmail.com
> >> > >
> >> > > wrote:
> >> > >
> >> > > > I advise (or not...depends on point of view) to stay that way
-
> >> because
> >> > > > when you activate write-back cache - live migrations will stop,
> and
> >> > this
> >> > > > makes *Enable maintenance mode (put host into maintenance)*
> >> impossible.
> >> > > >
> >> > > > I would perhaps suggest that there is documentation for "advanced
> >> > users"
> >> > > or
> >> > > > similar, that will say "it's possible to enable this and this
way
> >> via
> >> > DB
> >> > > > hack, but be warned on live migration consequences etc..." since
> >> this
> >> > > will
> >> > > > render more problems if people start using it.
> >> > > >
> >> > > > If you choose to do, let me know, I can write that (documentation)
> >> > > briefly.
> >> > > >
> >> > > > Not to mention it can be unsafe (power failure - less possible
I
> >> guess,
> >> > > but
> >> > > > rare kernel panic etc might have it's consequences I assume)
> >> > > >
> >> > > > It does indeed increase performance on NFS much, but not
> >> necessarily on
> >> > > > CEPH (if you are using lirbd cache on client side as explained
> >> above)
> >> > > >
> >> > > > On 20 February 2018 at 15:48, Rafael Weingärtner <
> >> > > > rafaelweingartner@gmail.com> wrote:
> >> > > >
> >> > > > > Yes. Weird enough, the code is using the value in the database
> if
> >> it
> >> > is
> >> > > > > provided there, but there is no easy way for users to change
> that
> >> > > > > configuration in the database. ¯\_(ツ)_/¯
> >> > > > >
> >> > > > > On Tue, Feb 20, 2018 at 11:45 AM, Andrija Panic <
> >> > > andrija.panic@gmail.com
> >> > > > >
> >> > > > > wrote:
> >> > > > >
> >> > > > > > So it seems that just passing the cachemode value to
API is
> not
> >> > > there,
> >> > > > or
> >> > > > > > somehow messedup, but deployVM process does read DB
values
> from
> >> > > > > > disk_offering table for sure, and applies it to XML
file for
> >> KVM.
> >> > > > > > This is above ACS 4.8.x.
> >> > > > > >
> >> > > > > >
> >> > > > > > On 20 February 2018 at 15:44, Andrija Panic <
> >> > andrija.panic@gmail.com
> >> > > >
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > > > I have edited the disk_offering table, in the
cache_mode
> just
> >> > enter
> >> > > > > > > "writeback". Stop and start VM, and it will pickup/inherit
> the
> >> > > > > cache_mode
> >> > > > > > > from it's parrent offering
> >> > > > > > > This also applies to Compute/Service offering,
again inside
> >> > > > > disk_offering
> >> > > > > > > table - just tested both
> >> > > > > > >
> >> > > > > > > i.e.
> >> > > > > > >
> >> > > > > > > UPDATE `cloud`.`disk_offering` SET `cache_mode`='writeback'
> >> WHERE
> >> > > > > > > `id`=102; # Compute Offering (Service offering)
> >> > > > > > > UPDATE `cloud`.`disk_offering` SET `cache_mode`='writeback'
> >> WHERE
> >> > > > > > > `id`=114; #data disk offering
> >> > > > > > >
> >> > > > > > > Before SQL:
> >> > > > > > >
> >> > > > > > > root@ix1-c7-4:~# virsh dumpxml i-2-10-VM | grep
cache -A2
> >> > > > > > >       <driver name='qemu' type='qcow2' cache='none'/>
> >> > > > > > >       <source file='/mnt/63a3ae7b-9ea9-3884-
> >> > > > > a772-1ea939ef6ec3/1b655159-
> >> > > > > > > ae10-41cf-8987-f1cfb47fe453'/>
> >> > > > > > >       <target dev='vda' bus='virtio'/>
> >> > > > > > > --
> >> > > > > > >       <driver name='qemu' type='qcow2' cache='none'/>
> >> > > > > > >       <source file='/mnt/63a3ae7b-9ea9-3884-
> >> > > > > a772-1ea939ef6ec3/09bdadcb-
> >> > > > > > > ec6e-4dda-b37b-17b1a749257f'/>
> >> > > > > > >       <target dev='vdb' bus='virtio'/>
> >> > > > > > > --
> >> > > > > > >
> >> > > > > > > STOP and START VM = after SQL
> >> > > > > > >
> >> > > > > > > root@ix1-c7-4:~# virsh dumpxml i-2-10-VM | grep
cache -A2
> >> > > > > > >       <driver name='qemu' type='qcow2' cache='writeback'/>
> >> > > > > > >       <source file='/mnt/63a3ae7b-9ea9-3884-
> >> > > > > a772-1ea939ef6ec3/1b655159-
> >> > > > > > > ae10-41cf-8987-f1cfb47fe453'/>
> >> > > > > > >       <target dev='vda' bus='virtio'/>
> >> > > > > > > --
> >> > > > > > >       <driver name='qemu' type='qcow2' cache='writeback'/>
> >> > > > > > >       <source file='/mnt/63a3ae7b-9ea9-3884-
> >> > > > > a772-1ea939ef6ec3/09bdadcb-
> >> > > > > > > ec6e-4dda-b37b-17b1a749257f'/>
> >> > > > > > >       <target dev='vdb' bus='virtio'/>
> >> > > > > > > --
> >> > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > On 20 February 2018 at 14:03, Rafael Weingärtner
<
> >> > > > > > > rafaelweingartner@gmail.com> wrote:
> >> > > > > > >
> >> > > > > > >> I have no idea how it can change the performance.
If you
> >> look at
> >> > > the
> >> > > > > > >> content of the commit you provided, it is
only the commit
> >> that
> >> > > > enabled
> >> > > > > > the
> >> > > > > > >> use of getCacheMode from disk offerings. However,
it is not
> >> > > exposing
> >> > > > > any
> >> > > > > > >> way to users to change that value/configuration
in the
> >> > database. I
> >> > > > > might
> >> > > > > > >> have missed it; do you see any API methods
that receive the
> >> > > > parameter
> >> > > > > > >> "cacheMode" and then pass this parameter to
a
> "diskOffering"
> >> > > object,
> >> > > > > and
> >> > > > > > >> then persist/update this object in the database?
> >> > > > > > >>
> >> > > > > > >> May I ask how are you guys changing the cacheMode
> >> configuration?
> >> > > > > > >>
> >> > > > > > >> On Tue, Feb 20, 2018 at 9:56 AM, Paul Angus
<
> >> > > > paul.angus@shapeblue.com
> >> > > > > >
> >> > > > > > >> wrote:
> >> > > > > > >>
> >> > > > > > >> > I'm working with some guys who are experimenting
with the
> >> > > setting
> >> > > > as
> >> > > > > > if
> >> > > > > > >> > definitely seems to change the performance
of data disks.
> >> It
> >> > > also
> >> > > > > > >> changes
> >> > > > > > >> > the XML of the VM which is created.
> >> > > > > > >> >
> >> > > > > > >> > p.s.
> >> > > > > > >> > I've found this commit;
> >> > > > > > >> >
> >> > > > > > >> > https://github.com/apache/clou
> >> dstack/commit/1edaa36cc68e845a
> >> > > > > > >> 42339d5f267d49
> >> > > > > > >> > c82343aefb
> >> > > > > > >> >
> >> > > > > > >> > so I've got something to investigate
now, but API
> >> > documentation
> >> > > > must
> >> > > > > > >> > definitely be askew.
> >> > > > > > >> >
> >> > > > > > >> >
> >> > > > > > >> >
> >> > > > > > >> > paul.angus@shapeblue.com
> >> > > > > > >> > www.shapeblue.com
> >> > > > > > >> > 53 Chandos Place, Covent Garden, London
 WC2N 4HSUK
> >> > > > > > >> > @shapeblue
> >> > > > > > >> >
> >> > > > > > >> >
> >> > > > > > >> >
> >> > > > > > >> >
> >> > > > > > >> > -----Original Message-----
> >> > > > > > >> > From: Rafael Weingärtner [mailto:rafaelweingartner@gmai
> >> l.com]
> >> > > > > > >> > Sent: 20 February 2018 12:31
> >> > > > > > >> > To: dev <dev@cloudstack.apache.org>
> >> > > > > > >> > Subject: Re: Caching modes
> >> > > > > > >> >
> >> > > > > > >> > This cache mode parameter does not exist
in
> >> > > > "CreateDiskOfferingCmd"
> >> > > > > > >> > command. I also checked some commits
from 2, 3, 4 and 5
> >> years
> >> > > ago,
> >> > > > > and
> >> > > > > > >> > this parameter was never there. If you
check the API in
> >> [1],
> >> > you
> >> > > > can
> >> > > > > > see
> >> > > > > > >> > that it is not an expected parameter.
Moreover, I do not
> >> see
> >> > any
> >> > > > use
> >> > > > > > of
> >> > > > > > >> > "setCacheMode" in the code (in case it
is updated by some
> >> > other
> >> > > > > > method).
> >> > > > > > >> > Interestingly enough, the code uses "getCacheMode".
> >> > > > > > >> >
> >> > > > > > >> > In summary, it is not a feature, and
it does not work. It
> >> > looks
> >> > > > like
> >> > > > > > >> some
> >> > > > > > >> > leftover from dark ages when people could
commit anything
> >> and
> >> > > then
> >> > > > > > they
> >> > > > > > >> > would just leave a half implementation
there in our code
> >> base.
> >> > > > > > >> >
> >> > > > > > >> > [1]
> >> > > > > > >> > https://cloudstack.apache.org/api/apidocs-4.11/apis/
> >> > > > > > >> > createDiskOffering.html
> >> > > > > > >> >
> >> > > > > > >> >
> >> > > > > > >> > On Tue, Feb 20, 2018 at 9:19 AM, Andrija
Panic <
> >> > > > > > andrija.panic@gmail.com
> >> > > > > > >> >
> >> > > > > > >> > wrote:
> >> > > > > > >> >
> >> > > > > > >> > > I can also assume that "cachemode"
as API parameter is
> >> not
> >> > > > > > supported,
> >> > > > > > >> > > since when creating data disk offering
via GUI also
> >> doesn't
> >> > > set
> >> > > > it
> >> > > > > > in
> >> > > > > > >> > DB/table.
> >> > > > > > >> > >
> >> > > > > > >> > > CM:    create diskoffering name=xxx
displaytext=xxx
> >> > > > > > storagetype=shared
> >> > > > > > >> > > disksize=1024 cachemode=writeback
> >> > > > > > >> > >
> >> > > > > > >> > > this also does not set cachemode
in table... my guess
> >> it's
> >> > not
> >> > > > > > >> > > implemented in API
> >> > > > > > >> > >
> >> > > > > > >> > > Let me know if I can help with any
testing here.
> >> > > > > > >> > >
> >> > > > > > >> > > Cheers
> >> > > > > > >> > >
> >> > > > > > >> > > On 20 February 2018 at 13:09, Andrija
Panic <
> >> > > > > > andrija.panic@gmail.com>
> >> > > > > > >> > > wrote:
> >> > > > > > >> > >
> >> > > > > > >> > > > Hi Paul,
> >> > > > > > >> > > >
> >> > > > > > >> > > > not helping directly answering
your question, but
> here
> >> are
> >> > > > some
> >> > > > > > >> > > > observations and "warning"
if client's are using
> >> > write-back
> >> > > > > cache
> >> > > > > > on
> >> > > > > > >> > > > KVM level
> >> > > > > > >> > > >
> >> > > > > > >> > > >
> >> > > > > > >> > > > I have (long time ago) tested
performance in 3
> >> > combinations
> >> > > > > (this
> >> > > > > > >> > > > was not really thorough testing
but a brief testing
> >> with
> >> > FIO
> >> > > > and
> >> > > > > > >> > > > random IO WRITE)
> >> > > > > > >> > > >
> >> > > > > > >> > > > - just CEPH rbd cache (on KVM
side)
> >> > > > > > >> > > >            i.e. [client]
> >> > > > > > >> > > >                  rbd cache
= true
> >> > > > > > >> > > >                  rbd cache
writethrough until flush =
> >> true
> >> > > > > > >> > > >                  #(this is
default 32MB per volume,
> >> afaik
> >> > > > > > >> > > >
> >> > > > > > >> > > > - just KMV write-back cache
(had to manually edit
> >> > > > disk_offering
> >> > > > > > >> > > > table to activate cache mode,
since when creating new
> >> disk
> >> > > > > > offering
> >> > > > > > >> > > > via GUI, the disk_offering
tables was NOT populated
> >> with
> >> > > > > > >> > > > "write-back" sertting/value
> >> > > > > > >> > > ! )
> >> > > > > > >> > > >
> >> > > > > > >> > > > - both CEPH and KVM write-back
cahce active
> >> > > > > > >> > > >
> >> > > > > > >> > > > My observations were like following,
but would be
> good
> >> to
> >> > > > > actually
> >> > > > > > >> > > confirm
> >> > > > > > >> > > > by someone else:
> >> > > > > > >> > > >
> >> > > > > > >> > > > - same performance with only
CEPH caching or with
> only
> >> KVM
> >> > > > > caching
> >> > > > > > >> > > > - a bit worse performance with
both CEPH and KVM
> >> caching
> >> > > > active
> >> > > > > > >> > > > (nonsense combination, I know...)
> >> > > > > > >> > > >
> >> > > > > > >> > > >
> >> > > > > > >> > > > Please keep in mind, that some
ACS functionality, KVM
> >> > > > > > >> > > > live-migrations on shared storage
(NFS/CEPH) are NOT
> >> > > supported
> >> > > > > > when
> >> > > > > > >> > > > you use KVM write-back cache,
since this is
> considered
> >> > > > "unsafe"
> >> > > > > > >> > migration, more info here:
> >> > > > > > >> > > > https://doc.opensuse.org/documentation/leap/
> >> > virtualization/
> >> > > > > > >> > > html/book.virt/
> >> > > > > > >> > > > cha.cachemodes.html#sec.cache.mode.live.migration
> >> > > > > > >> > > >
> >> > > > > > >> > > > or in short:
> >> > > > > > >> > > > "
> >> > > > > > >> > > > The libvirt management layer
includes checks for
> >> migration
> >> > > > > > >> > > > compatibility based on several
factors. If the guest
> >> > storage
> >> > > > is
> >> > > > > > >> > > > hosted on a clustered file
system, is read-only or is
> >> > marked
> >> > > > > > >> > > > shareable, then the cache mode
is ignored when
> >> determining
> >> > > if
> >> > > > > > >> > > > migration can be allowed. Otherwise
libvirt will not
> >> allow
> >> > > > > > migration
> >> > > > > > >> > > > unless the cache mode is set
to none. However, this
> >> > > > restriction
> >> > > > > > can
> >> > > > > > >> > > > be overridden with the “unsafe”
option to the
> migration
> >> > > APIs,
> >> > > > > > which
> >> > > > > > >> > > > is also supported by virsh,
as for example in
> >> > > > > > >> > > >
> >> > > > > > >> > > > virsh migrate --live --unsafe
> >> > > > > > >> > > > "
> >> > > > > > >> > > >
> >> > > > > > >> > > > Cheers
> >> > > > > > >> > > > Andrija
> >> > > > > > >> > > >
> >> > > > > > >> > > >
> >> > > > > > >> > > > On 20 February 2018 at 11:24,
Paul Angus <
> >> > > > > > paul.angus@shapeblue.com>
> >> > > > > > >> > > wrote:
> >> > > > > > >> > > >
> >> > > > > > >> > > >> Hi Wido,
> >> > > > > > >> > > >>
> >> > > > > > >> > > >> This is for KVM (with Ceph
backend as it happens),
> the
> >> > API
> >> > > > > > >> > > >> documentation is out of
sync with UI capabilities,
> so
> >> I'm
> >> > > > > trying
> >> > > > > > to
> >> > > > > > >> > > >> figure out if we
> >> > > > > > >> > > >> *should* be able to set
cacheMode for root disks.
> It
> >> > seems
> >> > > > to
> >> > > > > > make
> >> > > > > > >> > > quite a
> >> > > > > > >> > > >> difference to performance.
> >> > > > > > >> > > >>
> >> > > > > > >> > > >>
> >> > > > > > >> > > >>
> >> > > > > > >> > > >> paul.angus@shapeblue.com
> >> > > > > > >> > > >> www.shapeblue.com
> >> > > > > > >> > > >> 53 Chandos Place, Covent
Garden, London  WC2N 4HSUK
> >> > > > @shapeblue
> >> > > > > > >> > > >>
> >> > > > > > >> > > >>
> >> > > > > > >> > > >>
> >> > > > > > >> > > >>
> >> > > > > > >> > > >> -----Original Message-----
> >> > > > > > >> > > >> From: Wido den Hollander
[mailto:wido@widodh.nl]
> >> > > > > > >> > > >> Sent: 20 February 2018
09:03
> >> > > > > > >> > > >> To: dev@cloudstack.apache.org
> >> > > > > > >> > > >> Subject: Re: Caching modes
> >> > > > > > >> > > >>
> >> > > > > > >> > > >>
> >> > > > > > >> > > >>
> >> > > > > > >> > > >> On 02/20/2018 09:46 AM,
Paul Angus wrote:
> >> > > > > > >> > > >> > Hey guys,
> >> > > > > > >> > > >> >
> >> > > > > > >> > > >> > Can anyone shed any
light on write caching in
> >> > CloudStack?
> >> > > > > > >> >  cacheMode
> >> > > > > > >> > > >> is available through the
UI for data disks (but not
> >> root
> >> > > > > disks),
> >> > > > > > >> but
> >> > > > > > >> > not
> >> > > > > > >> > > >> documented as an API option
for data or root disks
> >> > > (although
> >> > > > is
> >> > > > > > >> > > documented
> >> > > > > > >> > > >> as a response for data
disks).
> >> > > > > > >> > > >> >
> >> > > > > > >> > > >>
> >> > > > > > >> > > >> What hypervisor?
> >> > > > > > >> > > >>
> >> > > > > > >> > > >> In case of KVM it's passed
down to XML which then
> >> passes
> >> > it
> >> > > > to
> >> > > > > > >> > Qemu/KVM
> >> > > > > > >> > > >> which then handles the
caching.
> >> > > > > > >> > > >>
> >> > > > > > >> > > >> The implementation varies
per hypervisor, so that
> >> should
> >> > be
> >> > > > the
> >> > > > > > >> > > question.
> >> > > > > > >> > > >>
> >> > > > > > >> > > >> Wido
> >> > > > > > >> > > >>
> >> > > > > > >> > > >>
> >> > > > > > >> > > >> > #huh?
> >> > > > > > >> > > >> >
> >> > > > > > >> > > >> > thanks
> >> > > > > > >> > > >> >
> >> > > > > > >> > > >> >
> >> > > > > > >> > > >> >
> >> > > > > > >> > > >> > paul.angus@shapeblue.com
> >> > > > > > >> > > >> > www.shapeblue.com
> >> > > > > > >> > > >> > 53 Chandos Place,
Covent Garden, London  WC2N
> 4HSUK
> >> > > > > @shapeblue
> >> > > > > > >> > > >> >
> >> > > > > > >> > > >> >
> >> > > > > > >> > > >> >
> >> > > > > > >> > > >>
> >> > > > > > >> > > >
> >> > > > > > >> > > >
> >> > > > > > >> > > >
> >> > > > > > >> > > > --
> >> > > > > > >> > > >
> >> > > > > > >> > > > Andrija Panić
> >> > > > > > >> > > >
> >> > > > > > >> > >
> >> > > > > > >> > >
> >> > > > > > >> > >
> >> > > > > > >> > > --
> >> > > > > > >> > >
> >> > > > > > >> > > Andrija Panić
> >> > > > > > >> > >
> >> > > > > > >> >
> >> > > > > > >> >
> >> > > > > > >> >
> >> > > > > > >> > --
> >> > > > > > >> > Rafael Weingärtner
> >> > > > > > >> >
> >> > > > > > >>
> >> > > > > > >>
> >> > > > > > >>
> >> > > > > > >> --
> >> > > > > > >> Rafael Weingärtner
> >> > > > > > >>
> >> > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > --
> >> > > > > > >
> >> > > > > > > Andrija Panić
> >> > > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > > --
> >> > > > > >
> >> > > > > > Andrija Panić
> >> > > > > >
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > --
> >> > > > > Rafael Weingärtner
> >> > > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > > --
> >> > > >
> >> > > > Andrija Panić
> >> > > >
> >> > >
> >> > >
> >> > >
> >> > > --
> >> > > Rafael Weingärtner
> >> > >
> >> >
> >> >
> >> >
> >> > --
> >> >
> >> > Andrija Panić
> >> >
> >>
> >>
> >>
> >> --
> >> Rafael Weingärtner
> >>
> >
> >
> >
> > --
> >
> > Andrija Panić
> >
>
>
>
> --
>
> Andrija Panić
>



-- 
Rafael Weingärtner

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message