cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chiradeep Vittal <chirade...@gmail.com>
Subject Re: Modern template hosting
Date Mon, 27 Feb 2017 18:08:16 GMT
Sounds workable. The downloader code in the SSVM won't follow redirects I
think.
https://github.com/apache/cloudstack/blob/5511065fc20787619d9cd0444a65a3155fc9c921/core/src/com/cloud/storage/template/HttpTemplateDownloader.java#L93
https://goo.gl/dSi0r5
 Might need to add
client.setRedirectStrategy(new LaxRedirectStrategy());

On Mon, Feb 27, 2017 at 9:57 AM, Will Stevens <wstevens@cloudops.com> wrote:

> We haven't opened a ticket yet because we don't have a strategy yet.
>
> What do you guys think of this:
> - We setup a new github repo in the 'apache' org which consists of a single
> file with a list of active/supported mirrors.
> - I write a small web server, distributed as a binary, which can be hosted
> by ASF Infra.  This web server will query the current list of mirrors and
> will select one and then do a 302 redirect to that mirror.
>
> The act of 'choosing' a mirror could be done in a number of ways.
> - If we want to define an order, then it could just try from the top of the
> list and work its way down.  It would curl the target to make sure it gets
> a 200 and if it does, it would do a 302 redirect.
> - Or, if we want to distribute the load across the mirrors, we could pick
> from the list randomly.  Again, doing a curl to verify the mirror is up and
> then doing a redirect.
> - If we want to get fancy, we could do a reverse IP lookup and try to match
> the requester with their closest geographical mirror.
>
> Thoughts?
>
> *Will STEVENS*
> Lead Developer
>
> <https://goo.gl/NYZ8KK>
>
> On Mon, Feb 27, 2017 at 12:46 PM, Chiradeep Vittal <chiradeepv@gmail.com>
> wrote:
>
> > What steps are needed to set up a mirror? What does Infra need to do? Has
> > anybody filed a ticket with Infra?
> >
> > On Sun, Feb 26, 2017 at 10:17 PM, Raja Pullela <
> > raja.pullela@accelerite.com>
> > wrote:
> >
> > > Hi will,
> > >
> > > I believe, we didn’t get to close ‘getting a mirror on Apache’ because
> we
> > > needed someone on the Apache Infra side to close this. BTW,
> > > cloudstack-apt.get.eu (I think Nux manages this?) has all/most of the
> > > content.  Once we can close on the Apache mirror for hosting the
> > content, I
> > > can help assist getting the content there.
> > >
> > > For now, we have replicated the download.cloud.com content to ‘
> > > s3.download.accelerite.com’.
> > > Also, we are working on a set of steps/procedure to help with this
> > > change.  I will update everyone in about a week’s time on the details.
> > >
> > > Best,
> > > Raja Pullela
> > > Engineering Team,
> > > Accelerite, 2055 Laurelwood Road,
> > > Santa Clara, CA, 95054
> > >
> > > On 2/24/17, 11:23 PM, "williamstevens@gmail.com on behalf of Will
> > > Stevens" <williamstevens@gmail.com on behalf of wstevens@cloudops.com>
> > > wrote:
> > >
> > > unfortunately the template mirror conversation got caught up in details
> > and
> > > nobody took the lead on implementing a solution.
> > >
> > > citrix has been pinging me every couple months to say 'dude, we need to
> > > remove the dependency on download.citrix.com', but i have not had the
> > > cycles to get in and solve the problem.  the shutdown of that is
> imminent
> > > right now, so we need to solve it asap.
> > >
> > >
> > >
> > > *Will STEVENS*
> > > Lead Developer
> > >
> > > <https://goo.gl/NYZ8KK>
> > >
> > > On Fri, Feb 24, 2017 at 12:38 PM, Paul Angus <paul.angus@shapeblue.com
> >
> > > wrote:
> > >
> > > > Hi Nathan,
> > > >
> > > > Ideally, if you put the template location in (or use a template
> defined
> > > > in)  test_data.py then the actual location can be overridden by
> anyone
> > > > testing.
> > > >
> > > > For Trillian, we've copied all of the templates that people have
> define
> > > to
> > > > a local repo and then replace the URLs in test_data.py to reduce
> > > bandwidth
> > > > use and download times.
> > > >
> > > > Ie:
> > > >
> > > >             "bootableIso":
> > > >                 {
> > > >                     "displaytext": "Test Bootable ISO",
> > > >                     "name": "testISO",
> > > >                     "bootable": True,
> > > >                     "ispublic": False,
> > > >                     "url": "{{ marvin_images_location
> > > > }}/TinyCore-current.iso",
> > > >                     "ostype": 'Other Linux (64-bit)',
> > > >                     "mode": 'HTTP_DOWNLOAD'
> > > >         },
> > > >
> > > >
> > > >
> > > > I thought that we had come up with a solution for download.cloud.com
> ,
> > by
> > > > having a mirrorlist hosted in Community Apache 'space' with anyone
> able
> > > to
> > > > out themselves forward as a mirror.
> > > > But I must admit I lost track of whether anyone made the requisite
> > > changes
> > > > in code....
> > > >
> > > >
> > > >
> > > >
> > > > Kind regards,
> > > >
> > > > Paul Angus
> > > >
> > > > paul.angus@shapeblue.com
> > > > www.shapeblue.com
> > > > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > > > @shapeblue
> > > >
> > > >
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
> > > > Behalf Of Will Stevens
> > > > Sent: 24 February 2017 16:30
> > > > To: dev@cloudstack.apache.org
> > > > Subject: Re: Modern template hosting
> > > >
> > > > this is a hard questions.  in general, we should be setting up a
> mirror
> > > on
> > > > some cloudstack/apache domain and then mirror to other provided
> > > templates.
> > > >
> > > > we MUST come up with a solution to deprecate 'download.cloud.com',
> > that
> > > > is going to be going away any day now.
> > > >
> > > > i don't know the right way to solve this to be honest, but if you
> have
> > > > ideas, i am willing to help.
> > > >
> > > >
> > > >
> > > > *Will STEVENS*
> > > > Lead Developer
> > > >
> > > > <https://goo.gl/NYZ8KK>
> > > >
> > > > On Fri, Feb 24, 2017 at 11:25 AM, Nathan Johnson <njohnson@ena.com>
> > > wrote:
> > > >
> > > > > So not to re-open a can of worms, but I’m in a situation where
I
> need
> > > > > to come up with a Marvin component test that depends on a template
> > > > > based on a kernel that’s relatively new, i.e., newer than Centos
> 5.3
> > /
> > > > Ubuntu 10.04 .
> > > > > I see openvm.eu has a suitable template (Ubuntu 16.0.4 for KVM),
> but
> > > > > from looking at the thread "Migrating CloudStack content from
> > > > > download.cloud.com” it looks like there is resistance to using
> this
> > at
> > > > > least for hosting system vm templates over concerns of neutrality.
> > > > > Would this be suitable for a component test?  If not, what is a
> > > “blessed”
> > > > > template location?
> > > > >
> > > > > Thanks in advance!
> > > > >
> > > > > Nathan Johnson
> > > > > R&D Engineer
> > > > >
> > > > >
> > > > >
> > > > > 618 Grassmere Park Drive, Suite 12
> > > > > Nashville, TN 37211
> > > > > General Office: 615-312-6000
> > > > >
> > > > > website | blog | support
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > DISCLAIMER
> > > ==========
> > > This e-mail may contain privileged and confidential information which
> is
> > > the property of Accelerite, a Persistent Systems business. It is
> intended
> > > only for the use of the individual or entity to which it is addressed.
> If
> > > you are not the intended recipient, you are not authorized to read,
> > retain,
> > > copy, print, distribute or use this message. If you have received this
> > > communication in error, please notify the sender and delete all copies
> of
> > > this message. Accelerite, a Persistent Systems business does not accept
> > any
> > > liability for virus infected mails.
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message