cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Will Stevens <wstev...@cloudops.com>
Subject Re: State of the S3 secondary storage
Date Tue, 19 Jul 2016 15:41:38 GMT
We are using Swift as secondary storage.  It also offers inter-zone
secondary storage.  We have had to do fixes to make it work correctly, but
we have pushed those fixes upstream so they are available in 4.9 (and some
fixes earlier than that).

Like Wido said, the staging NFS is still needed which is a real problem
with these implementations.  You end up copying the data many times and you
will run into problems if you have lots of customers doing scheduled
snapshots at the same time because you can fill your NFS staging area
unless you give it a lot of space to work with.

The object storage integrations are working, but they are not designed very
well at present.  You have code for both S3 and Swift basically just
separated with conditionals and much of the logic is shared.  This is very
brittle because making changes in one integration can (and historically
has) broken the other implementation.  This is a relatively big problem
because most people working with either Swift or S3 will only have the
ability to test one of them, so they don't really have the ability to test
if they have broken the other.  It does not help that there is Zero CI
coverage run for either the Swift or S3 integrations due to the difficulty
building and tearing down object storage environments on the fly.

All in all, yes they should be working, but expect a bit of a bumpy road.

Hope that helps,

*Will STEVENS*
Lead Developer

*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_

On Tue, Jul 19, 2016 at 11:26 AM, Wido den Hollander <wido@widodh.nl> wrote:

>
> > Op 19 juli 2016 om 14:38 schreef Nux! <nux@li.nux.ro>:
> >
> >
> > Hello,
> >
> > Could anyone summarise the state of S3 (and clones) for secondary
> storage?
> > I read at some point that it's the only secondary storage type supported
> inter-zone, so it would appeal from a DR pov, but not sure how mature it is
> or if there are any serious gotchas involved.
> >
>
> We have it running for our largest region (Amsterdam) with Ceph's RADOS
> Gateway as backend. We had some issues, but they seem to be fixed after our
> latest patches.
>
> You still need the staging NFS though, so that doesn't solve anything.
>
> Wido
>
> > Thanks
> > Lucian
> >
> > --
> > Sent from the Delta quadrant using Borg technology!
> >
> > Nux!
> > www.nux.ro
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message