cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nux! <...@li.nux.ro>
Subject Re: State of the S3 secondary storage
Date Tue, 19 Jul 2016 21:37:37 GMT
Glusterfs ;)
-- 
Sent from my mobile device. Please excuse my brevity.

On 19 July 2016 21:24:45 BST, Will Stevens <wstevens@cloudops.com> wrote:
>Oh ya.  Gotta love NFS syncing to servers halfway around the world.  :P
>
>*Will STEVENS*
>Lead Developer
>
>*CloudOps* *| *Cloud Solutions Experts
>420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>w cloudops.com *|* tw @CloudOps_
>
>On Tue, Jul 19, 2016 at 4:06 PM, Syed Ahmed <sahmed@cloudops.com>
>wrote:
>
>> Right, but is that an artificial limitation imposed by Cloudstack of
>is
>> there something that will break if we make NFS region wide? If not,
>we can
>> look at moving the NFS to a region wide storage.
>>
>> -Syed
>>
>> On Tue, Jul 19, 2016 at 3:54 PM, Will Stevens <wstevens@cloudops.com>
>> wrote:
>>
>> > Probably because ACS does not treat it as region wide, unlike the
>object
>> > storage implementations.
>> >
>> > *Will STEVENS*
>> > Lead Developer
>> >
>> > *CloudOps* *| *Cloud Solutions Experts
>> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>> > w cloudops.com *|* tw @CloudOps_
>> >
>> > On Tue, Jul 19, 2016 at 2:52 PM, Syed Ahmed <sahmed@cloudops.com>
>wrote:
>> >
>> > > Is there a reason why NFS cannot be region-wide?
>> > >
>> > > -Syed
>> > >
>> > > On Tue, Jul 19, 2016 at 2:08 PM, Nux! <nux@li.nux.ro> wrote:
>> > >
>> > > > Thanks for your input guys.
>> > > >
>> > > > I think I'll stay with NFS for now after your replies, but it'd
>be
>> > pretty
>> > > > sweet the day when we'll be having region wide secondary
>storage.
>> > > >
>> > > > I'll need to think of another way to provide DR, perhaps some
>> automatic
>> > > > snapshot+download of sorts.
>> > > >
>> > > > Lucian
>> > > >
>> > > > --
>> > > > Sent from the Delta quadrant using Borg technology!
>> > > >
>> > > > Nux!
>> > > > www.nux.ro
>> > > >
>> > > > ----- Original Message -----
>> > > > > From: "Syed Ahmed" <sahmed@cloudops.com>
>> > > > > To: dev@cloudstack.apache.org
>> > > > > Cc: "Nux!" <nux@li.nux.ro>, "Cloudstack Users List" <
>> > > > users@cloudstack.apache.org>
>> > > > > Sent: Tuesday, 19 July, 2016 17:06:52
>> > > > > Subject: Re: State of the S3 secondary storage
>> > > >
>> > > > > As Will put it, the implementation for using Object store as
>> > secondary
>> > > > > storage relies on using Staging NFS. You would also loose the
>> > > > functionality
>> > > > > to do differential snapshots. All your snapshots will
>basically be
>> > full
>> > > > > volume snapshots. So, if you have a lot of snapshots being
>taken at
>> > > > > scheduled intervals, you might end up with wasting a lot of
>space
>> on
>> > > your
>> > > > > S3. Not to mention all of this has to go through the NFS
>which can
>> > (an
>> > > in
>> > > > > our experience has) fill up preventing you from creating new
>VMs
>> > > (because
>> > > > > templates cannot be downloaded). However you do benefit from
>a
>> region
>> > > > wide
>> > > > > Image store. We've worked around the NFS problem by basically
>> having
>> > > > > multiple staging NFS stores.
>> > > > >
>> > > > > I have plans to make Object stores better integrated in
>Cloudstack.
>> > The
>> > > > > holy grail would be to avoid staging NFS completely (or
>atleast in
>> > the
>> > > > > upload path). But for now we have to live with the
>limitations.
>> > > > >
>> > > > > -Syed
>> > > > >
>> > > > >
>> > > > > On Tue, Jul 19, 2016 at 11:41 AM, Will Stevens <
>> > wstevens@cloudops.com>
>> > > > > wrote:
>> > > > >
>> > > > >> We are using Swift as secondary storage.  It also offers
>> inter-zone
>> > > > >> secondary storage.  We have had to do fixes to make it work
>> > correctly,
>> > > > but
>> > > > >> we have pushed those fixes upstream so they are available
in
>4.9
>> > (and
>> > > > some
>> > > > >> fixes earlier than that).
>> > > > >>
>> > > > >> Like Wido said, the staging NFS is still needed which is
a
>real
>> > > problem
>> > > > >> with these implementations.  You end up copying the data
>many
>> times
>> > > and
>> > > > you
>> > > > >> will run into problems if you have lots of customers doing
>> scheduled
>> > > > >> snapshots at the same time because you can fill your NFS
>staging
>> > area
>> > > > >> unless you give it a lot of space to work with.
>> > > > >>
>> > > > >> The object storage integrations are working, but they are
>not
>> > designed
>> > > > very
>> > > > >> well at present.  You have code for both S3 and Swift
>basically
>> just
>> > > > >> separated with conditionals and much of the logic is shared.
> This
>> > is
>> > > > very
>> > > > >> brittle because making changes in one integration can (and
>> > > historically
>> > > > >> has) broken the other implementation.  This is a relatively
>big
>> > > problem
>> > > > >> because most people working with either Swift or S3 will
>only have
>> > the
>> > > > >> ability to test one of them, so they don't really have the
>ability
>> > to
>> > > > test
>> > > > >> if they have broken the other.  It does not help that there
>is
>> Zero
>> > CI
>> > > > >> coverage run for either the Swift or S3 integrations due
to
>the
>> > > > difficulty
>> > > > >> building and tearing down object storage environments on
the
>fly.
>> > > > >>
>> > > > >> All in all, yes they should be working, but expect a bit
of
>a
>> bumpy
>> > > > road.
>> > > > >>
>> > > > >> Hope that helps,
>> > > > >>
>> > > > >> *Will STEVENS*
>> > > > >> Lead Developer
>> > > > >>
>> > > > >> *CloudOps* *| *Cloud Solutions Experts
>> > > > >> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>> > > > >> w cloudops.com *|* tw @CloudOps_
>> > > > >>
>> > > > >> On Tue, Jul 19, 2016 at 11:26 AM, Wido den Hollander <
>> > wido@widodh.nl>
>> > > > >> wrote:
>> > > > >>
>> > > > >> >
>> > > > >> > > Op 19 juli 2016 om 14:38 schreef Nux! <nux@li.nux.ro>:
>> > > > >> > >
>> > > > >> > >
>> > > > >> > > Hello,
>> > > > >> > >
>> > > > >> > > Could anyone summarise the state of S3 (and clones)
for
>> > secondary
>> > > > >> > storage?
>> > > > >> > > I read at some point that it's the only secondary
>storage type
>> > > > >> supported
>> > > > >> > inter-zone, so it would appeal from a DR pov, but not
sure
>how
>> > > mature
>> > > > it
>> > > > >> is
>> > > > >> > or if there are any serious gotchas involved.
>> > > > >> > >
>> > > > >> >
>> > > > >> > We have it running for our largest region (Amsterdam)
with
>> Ceph's
>> > > > RADOS
>> > > > >> > Gateway as backend. We had some issues, but they seem
to
>be
>> fixed
>> > > > after
>> > > > >> our
>> > > > >> > latest patches.
>> > > > >> >
>> > > > >> > You still need the staging NFS though, so that doesn't
>solve
>> > > anything.
>> > > > >> >
>> > > > >> > Wido
>> > > > >> >
>> > > > >> > > Thanks
>> > > > >> > > Lucian
>> > > > >> > >
>> > > > >> > > --
>> > > > >> > > Sent from the Delta quadrant using Borg technology!
>> > > > >> > >
>> > > > >> > > Nux!
>> > > > >> > > www.nux.ro
>> > > > >> >
>> > > >
>> > >
>> >
>>

Mime
  • Unnamed multipart/alternative (inline, 7-Bit, 0 bytes)
View raw message