Mailing-List: contact users-help@cloudstack.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@cloudstack.apache.org
MIME-Version: 1.0
Sender: williamstevens@gmail.com
In-Reply-To: <CADMzvUrt7z887o3eT_Kt-5=vv0HNOfQ5B6UjSezMXVzbcH3W8w@mail.gmail.com>
References: <1474354026.4435.1468931915063.JavaMail.zimbra@li.nux.ro>
 <567675001.1239.1468942011256@ox.pcextreme.nl> <CAC2panoBnW2BeCS0F5PahWDwXbXv0Jw3Pg8VO=03qadMx=+-aw@mail.gmail.com>
 <CADMzvUpzkDoMzFz_ztyxxjmbRLKaGWRYBJBMqae_xUBMFu37hg@mail.gmail.com>
 <1479696618.4912.1468951692141.JavaMail.zimbra@li.nux.ro> <CADMzvUrt7z887o3eT_Kt-5=vv0HNOfQ5B6UjSezMXVzbcH3W8w@mail.gmail.com>
From: Will Stevens <wstevens@cloudops.com>
Date: Tue, 19 Jul 2016 15:54:48 -0400
Message-ID: <CAC2pano5WWQeOpa1HjLwd7SS6Rt-T3y0EHWnPVLmXm0+9QOrKQ@mail.gmail.com>
Subject: Re: State of the S3 secondary storage
To: "dev@cloudstack.apache.org" <dev@cloudstack.apache.org>
Cc: "Nux!" <nux@li.nux.ro>, Cloudstack Users List <users@cloudstack.apache.org>
Content-Type: multipart/alternative; boundary=001a1143e69e1cc83b0538027519
archived-at: Tue, 19 Jul 2016 19:54:56 -0000

--001a1143e69e1cc83b0538027519
Content-Type: text/plain; charset=UTF-8

Probably because ACS does not treat it as region wide, unlike the object
storage implementations.

*Will STEVENS*
Lead Developer

*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_

On Tue, Jul 19, 2016 at 2:52 PM, Syed Ahmed <sahmed@cloudops.com> wrote:

> Is there a reason why NFS cannot be region-wide?
>
> -Syed
>
> On Tue, Jul 19, 2016 at 2:08 PM, Nux! <nux@li.nux.ro> wrote:
>
> > Thanks for your input guys.
> >
> > I think I'll stay with NFS for now after your replies, but it'd be pretty
> > sweet the day when we'll be having region wide secondary storage.
> >
> > I'll need to think of another way to provide DR, perhaps some automatic
> > snapshot+download of sorts.
> >
> > Lucian
> >
> > --
> > Sent from the Delta quadrant using Borg technology!
> >
> > Nux!
> > www.nux.ro
> >
> > ----- Original Message -----
> > > From: "Syed Ahmed" <sahmed@cloudops.com>
> > > To: dev@cloudstack.apache.org
> > > Cc: "Nux!" <nux@li.nux.ro>, "Cloudstack Users List" <
> > users@cloudstack.apache.org>
> > > Sent: Tuesday, 19 July, 2016 17:06:52
> > > Subject: Re: State of the S3 secondary storage
> >
> > > As Will put it, the implementation for using Object store as secondary
> > > storage relies on using Staging NFS. You would also loose the
> > functionality
> > > to do differential snapshots. All your snapshots will basically be full
> > > volume snapshots. So, if you have a lot of snapshots being taken at
> > > scheduled intervals, you might end up with wasting a lot of space on
> your
> > > S3. Not to mention all of this has to go through the NFS which can (an
> in
> > > our experience has) fill up preventing you from creating new VMs
> (because
> > > templates cannot be downloaded). However you do benefit from a region
> > wide
> > > Image store. We've worked around the NFS problem by basically having
> > > multiple staging NFS stores.
> > >
> > > I have plans to make Object stores better integrated in Cloudstack. The
> > > holy grail would be to avoid staging NFS completely (or atleast in the
> > > upload path). But for now we have to live with the limitations.
> > >
> > > -Syed
> > >
> > >
> > > On Tue, Jul 19, 2016 at 11:41 AM, Will Stevens <wstevens@cloudops.com>
> > > wrote:
> > >
> > >> We are using Swift as secondary storage.  It also offers inter-zone
> > >> secondary storage.  We have had to do fixes to make it work correctly,
> > but
> > >> we have pushed those fixes upstream so they are available in 4.9 (and
> > some
> > >> fixes earlier than that).
> > >>
> > >> Like Wido said, the staging NFS is still needed which is a real
> problem
> > >> with these implementations.  You end up copying the data many times
> and
> > you
> > >> will run into problems if you have lots of customers doing scheduled
> > >> snapshots at the same time because you can fill your NFS staging area
> > >> unless you give it a lot of space to work with.
> > >>
> > >> The object storage integrations are working, but they are not designed
> > very
> > >> well at present.  You have code for both S3 and Swift basically just
> > >> separated with conditionals and much of the logic is shared.  This is
> > very
> > >> brittle because making changes in one integration can (and
> historically
> > >> has) broken the other implementation.  This is a relatively big
> problem
> > >> because most people working with either Swift or S3 will only have the
> > >> ability to test one of them, so they don't really have the ability to
> > test
> > >> if they have broken the other.  It does not help that there is Zero CI
> > >> coverage run for either the Swift or S3 integrations due to the
> > difficulty
> > >> building and tearing down object storage environments on the fly.
> > >>
> > >> All in all, yes they should be working, but expect a bit of a bumpy
> > road.
> > >>
> > >> Hope that helps,
> > >>
> > >> *Will STEVENS*
> > >> Lead Developer
> > >>
> > >> *CloudOps* *| *Cloud Solutions Experts
> > >> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> > >> w cloudops.com *|* tw @CloudOps_
> > >>
> > >> On Tue, Jul 19, 2016 at 11:26 AM, Wido den Hollander <wido@widodh.nl>
> > >> wrote:
> > >>
> > >> >
> > >> > > Op 19 juli 2016 om 14:38 schreef Nux! <nux@li.nux.ro>:
> > >> > >
> > >> > >
> > >> > > Hello,
> > >> > >
> > >> > > Could anyone summarise the state of S3 (and clones) for secondary
> > >> > storage?
> > >> > > I read at some point that it's the only secondary storage type
> > >> supported
> > >> > inter-zone, so it would appeal from a DR pov, but not sure how
> mature
> > it
> > >> is
> > >> > or if there are any serious gotchas involved.
> > >> > >
> > >> >
> > >> > We have it running for our largest region (Amsterdam) with Ceph's
> > RADOS
> > >> > Gateway as backend. We had some issues, but they seem to be fixed
> > after
> > >> our
> > >> > latest patches.
> > >> >
> > >> > You still need the staging NFS though, so that doesn't solve
> anything.
> > >> >
> > >> > Wido
> > >> >
> > >> > > Thanks
> > >> > > Lucian
> > >> > >
> > >> > > --
> > >> > > Sent from the Delta quadrant using Borg technology!
> > >> > >
> > >> > > Nux!
> > >> > > www.nux.ro
> > >> >
> >
>

--001a1143e69e1cc83b0538027519--