cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Burwell <>
Subject Re: swift/S3 cloud-wide storage
Date Fri, 25 Jan 2013 02:53:14 GMT

All zones that are online when the template is added will download the
template simultaneously from the source.  The pull from S3 occurs when
a zone is added after a template has been downloaded from the source.


On Jan 24, 2013, at 9:15 PM, Marcus Sorensen <> wrote:

> So I'm supposed to see it pull down from S3 in the scenarios I've tried, or
> is it only ever between zones? Because my understanding did not match my
> tests.
> On Jan 24, 2013 7:03 PM, "John Burwell" <> wrote:
>> Marcus,
>> You are correct in your understanding of the feature.  Its primary use is
>> to synchronize templates across zones -- namely the circumstances where the
>> manual copy templates function would be invoked.  It also provides the
>> means to backup template, iso, and snapshot data to S3-based object store.
>> The design document [1] lays out this use case, the limitations, and some
>> high level thoughts on evolving the storage architecture to support a more
>> robust integration.  It is also why I named the feature S3-backed secondary
>> storage instead of S3 secondary storage ...
>> Thanks,
>> -John
>> [1]
>> On Jan 24, 2013, at 6:53 PM, Marcus Sorensen <> wrote:
>>> Thanks john.  Did you see the patch I sent?
>>> Also, I'm scratching my head trying to figure out the utility of this.
>>> I thought I knew what it would do, but it's not doing what I expected
>>> :-)  In my dev environment my templates get sucked into my S3, but I
>>> can't find a scenario where they come back, is it only for cross
>>> zones?
>>> For example, if I create a zone, define some templates, the templates
>>> go to S3. Then I add a new secondary storage, and the templates
>>> download from the internet URLs they originated from, rather than S3.
>>> Then I create a VM, create a template of that (no url to pull it from,
>>> right?), then the template gets sucked into S3, but I don't see it go
>>> to my other secondary storage, even if I remove the original secondary
>>> storage (in which case the template just goes to ready:no).
>>> I guess I just pictured this as sort of a backup system, so that if
>>> you lose your secondary storage, or create new secondary storage
>>> sources, that it would create consistency between them. I know also
>>> that it's supposed to help replicate between zones, but at this point
>>> it seems like that's the only function? Next I'll create a second zone
>>> with a separate secondary storage and see if that pulls down.
>>> On Thu, Jan 24, 2013 at 2:43 PM, John Burwell <>
>> wrote:
>>>> Marcus,
>>>> I can't speak for Swift, but the main reason there is no update current
>> for S3 was time.  I agree that the keys and timeouts should be adjustable,
>> but the implementation of S3 ran much longer than my employer expected, and
>> a compromise I had to make was matching the functionality of S3.  Also, I
>> am expected that S3 will be re-written in the next release to fit into the
>> new storage architecture, and the limitation will be removed then.
>>>> Thanks,
>>>> -John
>>>> On Jan 24, 2013, at 3:02 PM, Marcus Sorensen <>
>> wrote:
>>>>> What's the issue on this with never being able to reconfigure it
>>>>> again? If I change my key or something in the database, does that
>>>>> work, or is there some technical reason other than no UI set up? I
>>>>> don't understand how it works well enough to get why you can't add it
>>>>> after the zone creation, or change your credentials/S3 provider
>>>>> afterward.

View raw message