cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abhinandan Prateek <abhinandan.prat...@shapeblue.com>
Subject Re: Modern template hosting
Date Wed, 01 Mar 2017 09:39:08 GMT
+1

After seeing this discussion running in circles several times, I think we should at least
get started with the simplest option.




On 27/02/17, 11:54 PM, "Chiradeep Vittal" <chiradeepv@gmail.com> wrote:

>The alternative foolproof way is to simply not provide the automatic
>download.
>
>Just document the mirror list URL where the templates can be found. I don't
>know why people reject this option.
>
>https://shankerbalan.net/blog/seed-cloudstack-templates-offline/
>
>On Mon, Feb 27, 2017 at 10:15 AM, Will Stevens <williamstevens@gmail.com>
>wrote:
>
>> OK. Thanks for the heads up.
>>
>> On Feb 27, 2017 1:08 PM, "Chiradeep Vittal" <chiradeepv@gmail.com> wrote:
>>
>> > Sounds workable. The downloader code in the SSVM won't follow redirects I
>> > think.
>> > https://github.com/apache/cloudstack/blob/5511065fc20787619d9cd0444a65a3
>> > 155fc9c921/core/src/com/cloud/storage/template/
>> > HttpTemplateDownloader.java#L93
>> > https://goo.gl/dSi0r5
>> >  Might need to add
>> > client.setRedirectStrategy(new LaxRedirectStrategy());
>> >
>> > On Mon, Feb 27, 2017 at 9:57 AM, Will Stevens <wstevens@cloudops.com>
>> > wrote:
>> >
>> > > We haven't opened a ticket yet because we don't have a strategy yet.
>> > >
>> > > What do you guys think of this:
>> > > - We setup a new github repo in the 'apache' org which consists of a
>> > single
>> > > file with a list of active/supported mirrors.
>> > > - I write a small web server, distributed as a binary, which can be
>> > hosted
>> > > by ASF Infra.  This web server will query the current list of mirrors
>> and
>> > > will select one and then do a 302 redirect to that mirror.
>> > >
>> > > The act of 'choosing' a mirror could be done in a number of ways.
>> > > - If we want to define an order, then it could just try from the top of
>> > the
>> > > list and work its way down.  It would curl the target to make sure it
>> > gets
>> > > a 200 and if it does, it would do a 302 redirect.
>> > > - Or, if we want to distribute the load across the mirrors, we could
>> pick
>> > > from the list randomly.  Again, doing a curl to verify the mirror is up
>> > and
>> > > then doing a redirect.
>> > > - If we want to get fancy, we could do a reverse IP lookup and try to
>> > match
>> > > the requester with their closest geographical mirror.
>> > >
>> > > Thoughts?
>> > >
>> > > *Will STEVENS*
>> > > Lead Developer
>> > >
>> > > <https://goo.gl/NYZ8KK>
>> > >
>> > > On Mon, Feb 27, 2017 at 12:46 PM, Chiradeep Vittal <
>> chiradeepv@gmail.com
>> > >
>> > > wrote:
>> > >
>> > > > What steps are needed to set up a mirror? What does Infra need to
do?
>> > Has
>> > > > anybody filed a ticket with Infra?
>> > > >
>> > > > On Sun, Feb 26, 2017 at 10:17 PM, Raja Pullela <
>> > > > raja.pullela@accelerite.com>
>> > > > wrote:
>> > > >
>> > > > > Hi will,
>> > > > >
>> > > > > I believe, we didn’t get to close ‘getting a mirror on Apache’
>> > because
>> > > we
>> > > > > needed someone on the Apache Infra side to close this. BTW,
>> > > > > cloudstack-apt.get.eu (I think Nux manages this?) has all/most
of
>> > the
>> > > > > content.  Once we can close on the Apache mirror for hosting
the
>> > > > content, I
>> > > > > can help assist getting the content there.
>> > > > >
>> > > > > For now, we have replicated the download.cloud.com content to
‘
>> > > > > s3.download.accelerite.com’.
>> > > > > Also, we are working on a set of steps/procedure to help with
this
>> > > > > change.  I will update everyone in about a week’s time on the
>> > details.
>> > > > >
>> > > > > Best,
>> > > > > Raja Pullela
>> > > > > Engineering Team,
>> > > > > Accelerite, 2055 Laurelwood Road,
>> > > > > Santa Clara, CA, 95054
>> > > > >
>> > > > > On 2/24/17, 11:23 PM, "williamstevens@gmail.com on behalf of
Will
>> > > > > Stevens" <williamstevens@gmail.com on behalf of
>> > wstevens@cloudops.com>
>> > > > > wrote:
>> > > > >
>> > > > > unfortunately the template mirror conversation got caught up
in
>> > details
>> > > > and
>> > > > > nobody took the lead on implementing a solution.
>> > > > >
>> > > > > citrix has been pinging me every couple months to say 'dude,
we
>> need
>> > to
>> > > > > remove the dependency on download.citrix.com', but i have not
had
>> > the
>> > > > > cycles to get in and solve the problem.  the shutdown of that
is
>> > > imminent
>> > > > > right now, so we need to solve it asap.
>> > > > >
>> > > > >
>> > > > >
>> > > > > *Will STEVENS*
>> > > > > Lead Developer
>> > > > >
>> > > > > <https://goo.gl/NYZ8KK>
>> > > > >
>> > > > > On Fri, Feb 24, 2017 at 12:38 PM, Paul Angus <
>> > paul.angus@shapeblue.com
>> > > >
>> > > > > wrote:
>> > > > >
>> > > > > > Hi Nathan,
>> > > > > >
>> > > > > > Ideally, if you put the template location in (or use a template
>> > > defined
>> > > > > > in)  test_data.py then the actual location can be overridden
by
>> > > anyone
>> > > > > > testing.
>> > > > > >
>> > > > > > For Trillian, we've copied all of the templates that people
have
>> > > define
>> > > > > to
>> > > > > > a local repo and then replace the URLs in test_data.py to
reduce
>> > > > > bandwidth
>> > > > > > use and download times.
>> > > > > >
>> > > > > > Ie:
>> > > > > >
>> > > > > >             "bootableIso":
>> > > > > >                 {
>> > > > > >                     "displaytext": "Test Bootable ISO",
>> > > > > >                     "name": "testISO",
>> > > > > >                     "bootable": True,
>> > > > > >                     "ispublic": False,
>> > > > > >                     "url": "{{ marvin_images_location
>> > > > > > }}/TinyCore-current.iso",
>> > > > > >                     "ostype": 'Other Linux (64-bit)',
>> > > > > >                     "mode": 'HTTP_DOWNLOAD'
>> > > > > >         },
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > I thought that we had come up with a solution for
>> > download.cloud.com
>> > > ,
>> > > > by
>> > > > > > having a mirrorlist hosted in Community Apache 'space' with
>> anyone
>> > > able
>> > > > > to
>> > > > > > out themselves forward as a mirror.
>> > > > > > But I must admit I lost track of whether anyone made the
>> requisite
>> > > > > changes
>> > > > > > in code....
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > Kind regards,
>> > > > > >
>> > > > > > Paul Angus
>> > > > > >
>> > > > > > paul.angus@shapeblue.com
>> > > > > > www.shapeblue.com
>> > > > > > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
>> > > > > > @shapeblue
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > -----Original Message-----
>> > > > > > From: williamstevens@gmail.com [mailto:williamstevens@gmail.com]
>> > On
>> > > > > > Behalf Of Will Stevens
>> > > > > > Sent: 24 February 2017 16:30
>> > > > > > To: dev@cloudstack.apache.org
>> > > > > > Subject: Re: Modern template hosting
>> > > > > >
>> > > > > > this is a hard questions.  in general, we should be setting
up a
>> > > mirror
>> > > > > on
>> > > > > > some cloudstack/apache domain and then mirror to other provided
>> > > > > templates.
>> > > > > >
>> > > > > > we MUST come up with a solution to deprecate 'download.cloud.com
>> ',
>> > > > that
>> > > > > > is going to be going away any day now.
>> > > > > >
>> > > > > > i don't know the right way to solve this to be honest, but
if you
>> > > have
>> > > > > > ideas, i am willing to help.
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > *Will STEVENS*
>> > > > > > Lead Developer
>> > > > > >
>> > > > > > <https://goo.gl/NYZ8KK>
>> > > > > >
>> > > > > > On Fri, Feb 24, 2017 at 11:25 AM, Nathan Johnson <
>> njohnson@ena.com
>> > >
>> > > > > wrote:
>> > > > > >
>> > > > > > > So not to re-open a can of worms, but I’m in a situation
where
>> I
>> > > need
>> > > > > > > to come up with a Marvin component test that depends
on a
>> > template
>> > > > > > > based on a kernel that’s relatively new, i.e., newer
than
>> Centos
>> > > 5.3
>> > > > /
>> > > > > > Ubuntu 10.04 .
>> > > > > > > I see openvm.eu has a suitable template (Ubuntu 16.0.4
for
>> KVM),
>> > > but
>> > > > > > > from looking at the thread "Migrating CloudStack content
from
>> > > > > > > download.cloud.com” it looks like there is resistance
to using
>> > > this
>> > > > at
>> > > > > > > least for hosting system vm templates over concerns
of
>> > neutrality.
>> > > > > > > Would this be suitable for a component test?  If not,
what is a
>> > > > > “blessed”
>> > > > > > > template location?
>> > > > > > >
>> > > > > > > Thanks in advance!
>> > > > > > >
>> > > > > > > Nathan Johnson
>> > > > > > > R&D Engineer
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > 618 Grassmere Park Drive, Suite 12
>> > > > > > > Nashville, TN 37211
>> > > > > > > General Office: 615-312-6000
>> > > > > > >
>> > > > > > > website | blog | support
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > > DISCLAIMER
>> > > > > ==========
>> > > > > This e-mail may contain privileged and confidential information
>> which
>> > > is
>> > > > > the property of Accelerite, a Persistent Systems business. It
is
>> > > intended
>> > > > > only for the use of the individual or entity to which it is
>> > addressed.
>> > > If
>> > > > > you are not the intended recipient, you are not authorized to
read,
>> > > > retain,
>> > > > > copy, print, distribute or use this message. If you have received
>> > this
>> > > > > communication in error, please notify the sender and delete all
>> > copies
>> > > of
>> > > > > this message. Accelerite, a Persistent Systems business does
not
>> > accept
>> > > > any
>> > > > > liability for virus infected mails.
>> > > > >
>> > > >
>> > >
>> >
>>

abhinandan.prateek@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
  
 

Mime
View raw message