cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Syed Ahmed <sah...@cloudops.com>
Subject Re: State of the S3 secondary storage
Date Tue, 19 Jul 2016 18:52:42 GMT
Is there a reason why NFS cannot be region-wide?

-Syed

On Tue, Jul 19, 2016 at 2:08 PM, Nux! <nux@li.nux.ro> wrote:

> Thanks for your input guys.
>
> I think I'll stay with NFS for now after your replies, but it'd be pretty
> sweet the day when we'll be having region wide secondary storage.
>
> I'll need to think of another way to provide DR, perhaps some automatic
> snapshot+download of sorts.
>
> Lucian
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> ----- Original Message -----
> > From: "Syed Ahmed" <sahmed@cloudops.com>
> > To: dev@cloudstack.apache.org
> > Cc: "Nux!" <nux@li.nux.ro>, "Cloudstack Users List" <
> users@cloudstack.apache.org>
> > Sent: Tuesday, 19 July, 2016 17:06:52
> > Subject: Re: State of the S3 secondary storage
>
> > As Will put it, the implementation for using Object store as secondary
> > storage relies on using Staging NFS. You would also loose the
> functionality
> > to do differential snapshots. All your snapshots will basically be full
> > volume snapshots. So, if you have a lot of snapshots being taken at
> > scheduled intervals, you might end up with wasting a lot of space on your
> > S3. Not to mention all of this has to go through the NFS which can (an in
> > our experience has) fill up preventing you from creating new VMs (because
> > templates cannot be downloaded). However you do benefit from a region
> wide
> > Image store. We've worked around the NFS problem by basically having
> > multiple staging NFS stores.
> >
> > I have plans to make Object stores better integrated in Cloudstack. The
> > holy grail would be to avoid staging NFS completely (or atleast in the
> > upload path). But for now we have to live with the limitations.
> >
> > -Syed
> >
> >
> > On Tue, Jul 19, 2016 at 11:41 AM, Will Stevens <wstevens@cloudops.com>
> > wrote:
> >
> >> We are using Swift as secondary storage.  It also offers inter-zone
> >> secondary storage.  We have had to do fixes to make it work correctly,
> but
> >> we have pushed those fixes upstream so they are available in 4.9 (and
> some
> >> fixes earlier than that).
> >>
> >> Like Wido said, the staging NFS is still needed which is a real problem
> >> with these implementations.  You end up copying the data many times and
> you
> >> will run into problems if you have lots of customers doing scheduled
> >> snapshots at the same time because you can fill your NFS staging area
> >> unless you give it a lot of space to work with.
> >>
> >> The object storage integrations are working, but they are not designed
> very
> >> well at present.  You have code for both S3 and Swift basically just
> >> separated with conditionals and much of the logic is shared.  This is
> very
> >> brittle because making changes in one integration can (and historically
> >> has) broken the other implementation.  This is a relatively big problem
> >> because most people working with either Swift or S3 will only have the
> >> ability to test one of them, so they don't really have the ability to
> test
> >> if they have broken the other.  It does not help that there is Zero CI
> >> coverage run for either the Swift or S3 integrations due to the
> difficulty
> >> building and tearing down object storage environments on the fly.
> >>
> >> All in all, yes they should be working, but expect a bit of a bumpy
> road.
> >>
> >> Hope that helps,
> >>
> >> *Will STEVENS*
> >> Lead Developer
> >>
> >> *CloudOps* *| *Cloud Solutions Experts
> >> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> >> w cloudops.com *|* tw @CloudOps_
> >>
> >> On Tue, Jul 19, 2016 at 11:26 AM, Wido den Hollander <wido@widodh.nl>
> >> wrote:
> >>
> >> >
> >> > > Op 19 juli 2016 om 14:38 schreef Nux! <nux@li.nux.ro>:
> >> > >
> >> > >
> >> > > Hello,
> >> > >
> >> > > Could anyone summarise the state of S3 (and clones) for secondary
> >> > storage?
> >> > > I read at some point that it's the only secondary storage type
> >> supported
> >> > inter-zone, so it would appeal from a DR pov, but not sure how mature
> it
> >> is
> >> > or if there are any serious gotchas involved.
> >> > >
> >> >
> >> > We have it running for our largest region (Amsterdam) with Ceph's
> RADOS
> >> > Gateway as backend. We had some issues, but they seem to be fixed
> after
> >> our
> >> > latest patches.
> >> >
> >> > You still need the staging NFS though, so that doesn't solve anything.
> >> >
> >> > Wido
> >> >
> >> > > Thanks
> >> > > Lucian
> >> > >
> >> > > --
> >> > > Sent from the Delta quadrant using Borg technology!
> >> > >
> >> > > Nux!
> >> > > www.nux.ro
> >> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message