cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nux! <...@li.nux.ro>
Subject Re: Migrating CloudStack content from download.cloud.com
Date Tue, 17 May 2016 10:12:50 GMT
Hello,

I would solve the dilemma in a manner similar to how the CentOS mirror list works.

Create download.cloudstack.org or just use the main domain, upload in a subdir a text/json
file similar to the mirrorlist, in there we specify sha/md5 sums of available files as well
as multiple trustworthy links, such as cloudstack.apt-get.eu, its mirrors in coreix/shapeblue
and so on.

This way we keep control and trust inside the Apache Cloudstack domain name controlled by
Apache and PMC.

A bit of intelligence has to be added to Cloudstack so it can follow/retry multiple links
and check file integrity sums. This way we also get some redundancy/HA for download links.

HTH
Lucian

--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro

----- Original Message -----
> From: "Giles Sirett" <giles.sirett@shapeblue.com>
> To: dev@cloudstack.apache.org
> Cc: "Wido den Hollander" <wido@widodh.nl>
> Sent: Tuesday, 17 May, 2016 09:34:38
> Subject: RE: Migrating CloudStack content from download.cloud.com

> This is a MAJOR problem and needs to be figured out before the current download
> site is removed - @Raja - have Citrix committed to it being there another year
> ?
> 
> It is my understanding that Cloudstack relies on this download at initial
> build/startup and then periodically if people do things like build a new zone
> 
> So, users environments aren't going to stop if its pulled, but nobody would be a
> able to build/install cloudstack and it may cause issues as and when users make
> config changes
> 
> The problem is, for new builds, its hard coded : you start the  Sec Storage VM,
> CloudStack will try to download the built in template from download.cloud.com
> 
> My understanding is that its simple db  change to make once an environment is
> running (exisiting users) but *may* require a code patch to fix the initial
> install
> 
> #notideal
> 
> Really, this should be maintained by ASF/ the project as its key to the software
> working, however my understanding is it involves binaries which ASF is never
> keen on (and may be some 3rd party binaries) - so that just wont happen
> 
> 
> 
> Apt-get is controlled by Wido et al (for which we all owe him a massive thanks)
> - but you are right, it is a 3rd party dependency and arguably a SPOF.  But,
> if ASF wont host binaries, we're always going to have this issue.
> 
> So, we could patch the code to point somewhere else - but where ? - by
> definition its got to be something maintained by a 3rd party
> 
> 
> We had this debate some time ago when talking about the repositories. We
> (ShapeBlue) maintain a repo for our customers, which wev'e always opened up as
> public. However, there were concerns about having a company domain name (which
> I understand) - so apt-get became our preferred repo.
> 
> 
> Thoughts on how to fix this:
> 
> 1. if ASF would allow us to create the host entry download.cloudstack.org. Its
> under their domain, but with the site being with a 3rd party it could be
> allowed
> 
> 2. maybe Citrix could commit to pointing the current DNS entry to a new location
> (would be simplest but may be legally complex and relies on ongoing goodwill)
> 
> 3. make a mod to allow users to choose the download location on install. We can
> then maintain apt-get. To remove the SPOF, we could then get the templates
> copied to a number of 3rd party locations and allow people to choose which one
> they use
> 
> 
> 
> 
> Kind Regards
> Giles
> 
> 
> giles.sirett@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
> 
> 
> 
> -----Original Message-----
> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On Behalf Of
> Will Stevens
> Sent: 16 May 2016 18:59
> To: dev@cloudstack.apache.org
> Cc: Wido den Hollander <wido@widodh.nl>
> Subject: Re: Migrating CloudStack content from download.cloud.com
> 
> @Ian, yes, but I think there is more to it than that.  We can change it going
> forward and we can move everything from there over to a new domain.
> I believe the problem is with all of the existing ACS or CCP installs out there
> currently pointing at the download.cloud.com URL, so when it goes down, those
> installs have to be updated.
> 
> Maybe I don't understand exactly, but that is the impression I am getting from
> this thread.  Can this be confirmed?
> 
> *Will STEVENS*
> Lead Developer
> 
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw @CloudOps_
> 
> On Mon, May 16, 2016 at 1:38 PM, Ian Rae <irae@cloudops.com> wrote:
> 
>> Hey guys, who controls apt-get.eu and given the (rather concerning)
>> level of dependency on the operation of the download.cloud.com - would
>> it not make sense to use a domain that is under the control of party
>> whose alignment with CloudStack will not shift. I hesitate to
>> implicate the red tape of Apache Foundation, but perhaps the fabled
>> CloudStack alliance organization would be a good way to handle this
>> long term.
>>
>> On Mon, May 16, 2016 at 1:32 PM, Raja Pullela
>> <raja.pullela@accelerite.com> wrote:
>> > Paul,
>> >
>> > trust me, it won’t be go away overnight.  We are talking to Citrix
>> > and
>> requested them to keep it active for another year.
>> >
>> > Once we have the content copied over (I will work with Wido and
>> > verify
>> this) to cloudstack.apt-get.eu and update the documentation (user
>> guides update and create a wiki to show how an existing installation
>> can fixed to point to new URLs) we don’t have a dependency on
>> “download.cloud.com” –
>> right?  sorry, if I am missing something here?   Please let me know,
>> >
>> > best,
>> > Raja
>> > Senior Manager, Product Development
>> > Accelerite,
>> > 2055, Laurelwood Road,  Santa Clara, CA 95054, USA
>> > Phone: 1-408-216-7010,  www.accelerite.com, @accelerite
>> >
>> >
>> > On 5/16/16, 9:38 PM, "Paul Angus" <paul.angus@shapeblue.com> wrote:
>> >>Thanks Raja,
>> >>
>> >>May I humbly suggest that someone at Accelerite speaks to Citrix
>> >>about
>> how long the content will remain on download.cloud.com?
>> >>Simply losing the content on download.cloud.com overnight will make
>> everyone look bad
>> >>
>> >>Kind regards,
>> >>
>> >>Paul Angus
>> >>
>> >>paul.angus@shapeblue.com
>> >>www.shapeblue.com
>> >>53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
>> >>
>> >>-----Original Message-----
>> >>From: Raja Pullela [mailto:raja.pullela@accelerite.com]
>> >>Sent: 16 May 2016 16:44
>> >>To: dev@cloudstack.apache.org
>> >>Subject: Re: Migrating CloudStack content from download.cloud.com
>> >>
>> >>Paul,
>> >>
>> >>Yes, you are right, this change impacts both CloudStack and
>> CloudPlatform customers.  Not sure, if I understand you correctly, the
>> move away from “download.cloud.com” is permanent.  I don’t know how
>> long the content on download.cloud.com will be up/available.
>> >>
>> >>Based on Wido’s availability, we can get rest of the content to
>> cloudstack.apt-get.eu, hopefully, by this week ?    BTW, the content on
>> download.cloud.com is left as is and will stay as is until Citrix
>> decides on what to do with it.
>> >>
>> >>Best,
>> >>Raja
>> >>Senior Manager, Product Development
>> >>Accelerite,
>> >>2055, Laurelwood Road,  Santa Clara, CA 95054, USA
>> >>Phone: 1-408-216-7010,  www.accelerite.com, @accelerite
>> >>
>> >>
>> >>On 5/16/16, 6:36 PM, "Paul Angus" <paul.angus@shapeblue.com> wrote:
>> >>
>> >>>Raja,
>> >>>
>> >>>As this effects every copy of CloudStack and CloudPlatform there
>> >>>ever
>> was.  Please may I ask how long we have to manage a migration away
>> from download.cloud.com?
>> >>>
>> >>>
>> >>>
>> >>>Kind regards,
>> >>>
>> >>>Paul Angus
>> >>>
>> >>>Regards,
>> >>>
>> >>>Paul Angus
>> >>>
>> >>>paul.angus@shapeblue.com
>> >>>www.shapeblue.com
>> >>>53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
>> >>>
>> >>>-----Original Message-----
>> >>>From: Raja Pullela [mailto:raja.pullela@accelerite.com]
>> >>>Sent: 16 May 2016 13:53
>> >>>To: dev@cloudstack.apache.org
>> >>>Subject: Re: Migrating CloudStack content from download.cloud.com
>> >>>
>> >>>Hi Paul,
>> >>>
>> >>>Thank you for the additional file details and I will request wido
>> >>>to
>> copy these as well.
>> >>>
>> >>>BTW, I agree with you there is every copy of CloudStack out there
>> >>>and
>> reason why I have requested Wido to download all the old files as well.
>> Looks like we missed few and get these uploaded as well.
>> >>>
>> >>>Please note that Citrix owns the “download.cloud.com” address and
>> >>>we
>> have to get all the CloudStack related content from there.  Once we
>> confirm on the content move Citrix may choose to remove the content
>> from the location.
>> >>>
>> >>>That said, I will get in touch with Wido again and do a complete
>> >>>copy
>> such that we have everything.
>> >>>
>> >>>Best,
>> >>>Raja
>> >>>Senior Manager, Product Development Accelerite, 2055, Laurelwood
>> >>>Road,  Santa Clara, CA 95054, USA
>> >>>Phone: 1-408-216-7010,  www.accelerite.com, @accelerite
>> >>>
>> >>>On 5/14/16, 7:12 AM, "Paul Angus" <paul.angus@shapeblue.com> wrote:
>> >>>>Raja,
>> >>>>
>> >>>>I've just picked up on this.
>> >>>>
>> >>>>There are a number of URLs which are hardcoded into the CloudStack
>> database for the small built-in templates.  These are in just about
>> every copy of CloudStack out there.
>> >>>>
>> >>>>http://download.cloud.com/templates/builtin/f59f18fb-ae94-4f97-afd
>> >>>>2-f8
>> >>>>4
>> >>>>755767aca.vhd.bz2
>> >>>>http://download.cloud.com/releases/2.2.0/eec2209b-9875-3c8d-92be-c
>> >>>>001b
>> >>>>d
>> >>>>8a0faf.qcow2.bz2
>> >>>>http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz
>> >>>>2
>> >>>>http://download.cloud.com/releases/4.3/centos6_4_64bit.vhd.bz2
>> >>>>http://download.cloud.com/releases/2.2.0/CentOS5.3-x86_64.ova
>> >>>>http://download.cloud.com/templates/builtin/centos-7-x86_64.tar.gz
>> >>>>
>> >>>>
>> >>>>These will need copying and the CloudStack code updating to the
>> >>>>new
>> location.
>> >>>>
>> >>>>
>> >>>>I don't think that you can deprecate download.cloud.com or it's
>> contents for some time, without causing a lot of issues for users.
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>Hi,
>> >>>>
>> >>>>
>> >>>>
>> >>>>Short Term - all the content has moved to cloudstack.apt-get.eu
>> >>>>
>> >>>>
>> >>>>
>> >>>>Which sections to be modified?
>> >>>>
>> >>>>
>> >>>>
>> >>>>The Install Guide needs to be modified.  Sample links/details are
>> >>>>mentioned below for 4.6 and same needs to be updated for other
>> >>>>Released versions of Cloudstack documentation:
>> >>>>
>> >>>>http://docs.cloudstack.apache.org/projects/cloudstack-installation
>> >>>>/en/
>> >>>>4 .6/management-server/#downloading-vhd-util
>> >>>>
>> >>>>Section - Downloading vhd-util¶
>> >>>>
>> >>>>
>> >>>>
>> >>>>What to do modified ?
>> >>>>
>> >>>>modify URL from
>> >>>>http://download.cloud.com.s3.amazonaws.com/tools/vhd-util to
>> >>>>http://cloudstack.apt-get.eu/tools/vhd-util
>> >>>>
>> >>>>
>> >>>>
>> >>>>@Rajsekhar can you please update the sections ?
>> >>>>
>> >>>>
>> >>>>
>> >>>>Thanks
>> >>>>
>> >>>>Raja
>> >>>>
>> >>>>Senior Manager, Product Development
>> >>>>
>> >>>>Accelerite,
>> >>>>
>> >>>>2055, Laurelwood Road,  Santa Clara, CA 95054, USA
>> >>>>
>> >>>>Phone: 1-408-216-7010,  www.accelerite.com, @accelerite
>> >>>>
>> >>>>
>> >>>>Kind regards,
>> >>>>
>> >>>>Paul Angus
>> >>>>
>> >>>>
>> >>>>Regards,
>> >>>>
>> >>>>Paul Angus
>> >>>>
>> >>>>paul.angus@shapeblue.com
>> >>>>www.shapeblue.com
>> >>>>53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>DISCLAIMER
>> >>>==========
>> >>>This e-mail may contain privileged and confidential information
>> >>>which
>> is the property of Accelerite, a Persistent Systems business. It is
>> intended only for the use of the individual or entity to which it is
>> addressed. If you are not the intended recipient, you are not
>> authorized to read, retain, copy, print, distribute or use this
>> message. If you have received this communication in error, please
>> notify the sender and delete all copies of this message. Accelerite, a
>> Persistent Systems business does not accept any liability for virus infected
>> mails.
>> >>
>> >>
>> >>
>> >>
>> >>DISCLAIMER
>> >>==========
>> >>This e-mail may contain privileged and confidential information
>> >>which is
>> the property of Accelerite, a Persistent Systems business. It is
>> intended only for the use of the individual or entity to which it is
>> addressed. If you are not the intended recipient, you are not
>> authorized to read, retain, copy, print, distribute or use this
>> message. If you have received this communication in error, please
>> notify the sender and delete all copies of this message. Accelerite, a
>> Persistent Systems business does not accept any liability for virus infected
>> mails.
>> >
>> >
>> >
>> >
>> > DISCLAIMER
>> > ==========
>> > This e-mail may contain privileged and confidential information
>> > which is
>> the property of Accelerite, a Persistent Systems business. It is
>> intended only for the use of the individual or entity to which it is
>> addressed. If you are not the intended recipient, you are not
>> authorized to read, retain, copy, print, distribute or use this
>> message. If you have received this communication in error, please
>> notify the sender and delete all copies of this message. Accelerite, a
>> Persistent Systems business does not accept any liability for virus infected
>> mails.
>>
>>
>>
>> --
>> Ian Rae
>> CEO | PDG
>> c: 514.944.4008
>>
>> CloudOps | Cloud Infrastructure and Networking Solutions
>> www.cloudops.com | 420 rue Guy | Montreal | Canada | H3J 1S6

Mime
View raw message