cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erik Weber <terbol...@gmail.com>
Subject Re: Modern template hosting
Date Fri, 03 Mar 2017 23:19:07 GMT
IMHO; implement it as simple as possible, use DNS RR and assume/expect
mirror hosts to use the same path.


Yes I know; DNS RR isn't bulletproof, but as already mentioned in this
thread - mirrors rarely go down and they aren't used all that often
(primarily seeding).
We can keep a low TTL on the record so that we're able to remove a
mirror that goes down for a significant period of time (more than
hours).

And yes I know, some operators might find it a burden having to set up
a new vhost with another document root, but tbh if they don't care
enough to do it we'll manage without that mirror.

-- 
Erik

On Sat, Mar 4, 2017 at 12:09 AM, Will Stevens <williamstevens@gmail.com> wrote:
> I agree with Paul. Look at the list of things they have to
> learn/master/care about. We don't want to add to that list, we want to
> remove from that list. Make the project/product more accessible. Reduce the
> barrier to entry...
>
> On Mar 3, 2017 5:50 PM, "Chiradeep Vittal" <chiradeepv@gmail.com> wrote:
>
>> Seriously?
>> apt-get install apache2
>>
>> To install CloudStack, they need to know
>> - DB installation, perhaps some SQL
>> - VLAN configuration on their switches
>> - Ins-and-outs between port forwarding, static ips,
>> - NFS
>> - package management
>> - VHDs, qcow2. vmdk
>> - hyperviosrs
>> - and on and on.
>>
>> If they can figure all that out and not figure out a web server, they can
>> always come to the mailing list.
>>
>>
>>
>>
>> On Fri, Mar 3, 2017 at 1:45 PM, Paul Angus <paul.angus@shapeblue.com>
>> wrote:
>>
>> > The issue is not with supporting highly experienced cloud operators. It
>> is
>> > an issue for new users and other 'relatively' inexperienced operators.
>> >
>> > I have helped enough newbies and cloud operators who have been running
>> > their cloud for a while to know that the barriers to entry are too high
>> as
>> > they are. And telling anyone that they need to create a web server so
>> that
>> > they can add their initial template to get started or in order to create
>> a
>> > new zone, just isn't going to fly.
>> >
>> > I'm a huge advocate of the 'download.cloudstack.org' endpoint which the
>> > community can add/remove mirrors to or from for system VMs or built-in
>> > templates.  Can the same system be used for binary repos ? although there
>> > is an added complication of redist vs no-redist there...
>> >
>> >
>> >
>> >
>> >
>> >
>> > paul.angus@shapeblue.com
>> > www.shapeblue.com
>> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
>> > @shapeblue
>> >
>> >
>> >
>> >
>> > -----Original Message-----
>> > From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
>> > Sent: 03 March 2017 18:28
>> > To: dev <dev@cloudstack.apache.org>
>> > Subject: Re: Modern template hosting
>> >
>> > I do feel like this is early optimization. Mirrors rarely fail. I'd
>> expect
>> > a single web server hosted on Apache Infra without any monitors to fail
>> > more often than a mirror. We already expect Wido's systemvm repository to
>> > be up all the time. And it has been. Similarly, I don't believe Nux's
>> > repository has ever been down. And if Accelerite wants to host on S3,
>> that
>> > one is pretty solid as well.
>> >
>> > This is an infrequent operation in a cloud. After the cloud is installed,
>> > the download servers are only needed for a new zone. If we trust the user
>> > to run a cloud, surely he/she can run a web server to serve some built-in
>> > templates. And if her cloud is successful, she needs to figure out how to
>> > host her templates anyway and not rely on 3rd parties.
>> >
>> >
>> > On Fri, Mar 3, 2017 at 9:31 AM, Will Stevens <wstevens@cloudops.com>
>> > wrote:
>> >
>> > > 1) If the legacy implementations do not support redirects, that does
>> > > cause a problem.  A potential solution in that case is to have the web
>> > > server actually proxy the download, but that is not ideal and I would
>> > > like to avoid it if possible.  Thanks for bringing that up Chiradeep.
>> > >
>> > > 2) I think we need to have a single URL which people can target.  Once
>> > > they make the switch to the new URL, we want the implementation to be
>> > > able to handle mirror failures without affecting the end client.  We
>> > > want to avoid the situation where an ACS user will ever have to change
>> > > this URL more than once.  Mirror failures SHOULD NOT affect the ACS
>> > > users assuming there is still at least one mirror who can serve the
>> > requested resource.
>> > >
>> > > These are obviously my personal opinions and others will probably have
>> > > differing opinions.
>> > >
>> > > *Will STEVENS*
>> > > Lead Developer
>> > >
>> > > <https://goo.gl/NYZ8KK>
>> > >
>> > > On Fri, Mar 3, 2017 at 12:23 PM, Chiradeep Vittal
>> > > <chiradeepv@gmail.com>
>> > > wrote:
>> > >
>> > > > 1. If you are targeting legacy installations, they are not able to
>> > > > follow redirects. The line of code that added this capability was
>> > > > added on
>> > > 11/16.
>> > > > 2. If you trust the users to edit the database to change the URL,
>> > > > you can trust them to change it to anything. Just document a known
>> > > > list of good template locations.
>> > > >
>> > > > On Fri, Mar 3, 2017 at 8:13 AM, Will Stevens <wstevens@cloudops.com>
>> > > > wrote:
>> > > >
>> > > > > So the main issue I see with this is the following.
>> > > > >
>> > > > > This implementation is designed to target legacy installations
>> > > > > which
>> > > will
>> > > > > be affected when download.cloud.com disappears.  These people will
>> > > need
>> > > > to
>> > > > > make a DB change to replace the 'download.cloud.com' with some
>> > > > > other
>> > > url
>> > > > > (maybe 'download.cloudstack.org').  Once the DB has been updated,
>> > > > > we
>> > > can
>> > > > > not expect anything else of the client, they need to be able to
>> > > > > just continue operation as they were without any need to rebuild
>> > > > > or upgrade their ACS.
>> > > > >
>> > > > > If we try to force the decision to the client, then we break the
>> > > > > legacy implementations.  Since the SSVM is likely going to be the
>> > > > > client in
>> > > some
>> > > > > cases, and since it does not already have the logic to handle the
>> > > > > 300 approach correctly, I feel like it is not a viable solution
>> > > > > for the
>> > > > legacy
>> > > > > deployments.
>> > > > >
>> > > > > In addition to that.  I don't think the client is in any better
>> > > position
>> > > > to
>> > > > > make the mirror decision than I am.  I am likely able to make a
>> > > > > more 'educated' decision than the client would be able to because
>> > > > > I can do different tests on the endpoint(s) before making a final
>> > decision.
>> > > > >
>> > > > > Does this make sense or am I missing something obvious?
>> > > > >
>> > > > > *Will STEVENS*
>> > > > > Lead Developer
>> > > > >
>> > > > > <https://goo.gl/NYZ8KK>
>> > > > >
>> > > > > On Fri, Mar 3, 2017 at 10:48 AM, Rafael Weingärtner <
>> > > > > rafaelweingartner@gmail.com> wrote:
>> > > > >
>> > > > > > I think I understood you, but I did not understand you.
>> > > > > >
>> > > > > > Let me see if I can create an example to illustrate how we could
>> > > > > > do
>> > > > using
>> > > > > > the HTTP 300 code.
>> > > > > >
>> > > > > > The HTTP 300 code indicates clients that he/she/it has multiple
>> > > choices
>> > > > > to
>> > > > > > access the requested resource. If the request is a HEAD, this is
>> > > > > > the
>> > > > only
>> > > > > > thing that the server returns. If the request if a GET, then the
>> > > server
>> > > > > > returns the choices for the client. The response type is defined
>> > > > > > on
>> > > the
>> > > > > > server based on the Content-Type, user agent and others.
>> > > > > >
>> > > > > > Let’s say the server receives a GET request and informs the
>> > > > content-type
>> > > > > as
>> > > > > > JSON (application/JSON). Your application could get the mirrors
>> > > > > > list,
>> > > > do
>> > > > > > the appending process for the requested resource path, and then
>> > > return
>> > > > a
>> > > > > > JSON array with the possible mirror the client can use. Then,
>> > > > > > the
>> > > > client
>> > > > > > gets this list and does the processing required to select a
>> mirror.
>> > > > > >
>> > > > > > Now let’s say that a human access the link using a browser.
>> > > > > > Then, the server should receive something like (text/plain) as
>> > > > > > the
>> > > content-type;
>> > > > I
>> > > > > > think depending on the browser this field may differ a little
>> > > > > > (it
>> > > would
>> > > > > > require some checking). Anyways, the server detects that it is a
>> > > > “human”
>> > > > > > requesting the resource, then we could serve an HTML page with a
>> > > > > Javascript
>> > > > > > that uses the mirror list. This Javascript could do some
>> > > > > > checking and choose the best mirror for that specific.
>> > > > > >
>> > > > > > Why do I think this approach is interesting?
>> > > > > > The application you developed would be used only to retrieve
>> > > > > > valid
>> > > and
>> > > > > > trusted sources of system VM images (a trusted repository for
>> > > > > > mirrors
>> > > > > that
>> > > > > > we as a community vouch for).  Also, we shift the decision
>> > > > > > process regarding mirrors from the server to the client. Then,
>> > > > > > it is up to
>> > > > > clients
>> > > > > > to select mirrors, and not up to us (or some of our
>> > implementations).
>> > > > > >
>> > > > > > Did this help?
>> > > > > > BTW: I am not saying I am against the way you proposed, which
>> > > > > > would
>> > > > work
>> > > > > > fine. It is merely a suggestion using a different perspective.
>> > > > > >
>> > > > > > On Fri, Mar 3, 2017 at 10:23 AM, Will Stevens <
>> > > > williamstevens@gmail.com>
>> > > > > > wrote:
>> > > > > >
>> > > > > > > @rafael: in general, I don't think the client should ever be
>> > > making a
>> > > > > > > choice about a mirror. We have to assume we are working with a
>> > > > scripted
>> > > > > > > application and anything hard coded in that implementation on
>> > > > > > > the
>> > > > > client
>> > > > > > > side is a risk.
>> > > > > > >
>> > > > > > > I may not be understanding the use of the 300 approach, so I
>> > > > > > > will
>> > > > > > research
>> > > > > > > it to see if I can make it fit.
>> > > > > > >
>> > > > > > > On Mar 3, 2017 9:59 AM, "Rafael Weingärtner" <
>> > > > > > rafaelweingartner@gmail.com>
>> > > > > > > wrote:
>> > > > > > >
>> > > > > > > Will great job.
>> > > > > > >
>> > > > > > > I had the same doubt as Daan.
>> > > > > > >
>> > > > > > > Have you considered using HTTP 300 working mode? Then, we
>> > > > > > > could let
>> > > > the
>> > > > > > > client decide which mirror is the best(closest?). Section
>> > “10.3.1”
>> > > of
>> > > > > [1]
>> > > > > > > talks about it; this would put the pressure on deciding upon a
>> > > mirror
>> > > > > on
>> > > > > > > the client side, but I think it would be fair.
>> > > > > > >
>> > > > > > > [1] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html
>> > > > > > >
>> > > > > > > PS: You really like very short variables names!
>> > > > > > >
>> > > > > > > On Fri, Mar 3, 2017 at 1:24 AM, Daan Hoogland <
>> > > > > > daan.hoogland@shapeblue.com
>> > > > > > > >
>> > > > > > > wrote:
>> > > > > > >
>> > > > > > > > Nice little thing Will,
>> > > > > > > >
>> > > > > > > > One question: if I read the code correctly it ‘go’es and
>> > > > > > > > tries
>> > > all
>> > > > > > > mirrors
>> > > > > > > > at once and whichever responses first is redirected to the
>> > > client.
>> > > > > This
>> > > > > > > > might well be the same every time. This might be the one
>> > > > > > > > closest
>> > > to
>> > > > > the
>> > > > > > > > mirror302 and not to the client. Is that correct or did I
>> > > > > > > > miss a
>> > > > > > weighing
>> > > > > > > > algorithm hidden in there?
>> > > > > > > >
>> > > > > > > > Good coding,
>> > > > > > > >
>> > > > > > > > On 03/03/17 00:23, "Will Stevens" <swill@apache.org> wrote:
>> > > > > > > >
>> > > > > > > >     Hey All,
>> > > > > > > >     Please review this repo and tell me what you think:
>> > > > > > > >     https://github.com/swill/mirror302
>> > > > > > > >
>> > > > > > > >     Let me know if you have questions or you would like me
>> > > > > > > > to
>> > > make
>> > > > > > > changes
>> > > > > > > >     before I bring this topic up with ASF Infra and open a
>> > > > > > > > ticket
>> > > > to
>> > > > > > get
>> > > > > > > > this
>> > > > > > > >     implemented.
>> > > > > > > >
>> > > > > > > >     Cheers,
>> > > > > > > >
>> > > > > > > >     *Will Stevens*
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >     On Wed, Mar 1, 2017 at 11:37 AM, Will Stevens <
>> > > > > > wstevens@cloudops.com
>> > > > > > > >
>> > > > > > > > wrote:
>> > > > > > > >
>> > > > > > > >     > I am building a short term solution right now.  I
>> > > > > > > > hopefully
>> > > > > will
>> > > > > > > > have the
>> > > > > > > >     > building blocks in place by the weekend so we can
>> > > > > > > > start
>> > > > working
>> > > > > > > with
>> > > > > > > > Infra
>> > > > > > > >     > to get it in place.
>> > > > > > > >     >
>> > > > > > > >     > We will have a web server which we can point something
>> > like
>> > > > > > > >     > downloads.cloudstack.org (or whatever url) at.  Then
>> > > > > > > > we
>> > > will
>> > > > > > have
>> > > > > > > > an ASF
>> > > > > > > >     > repo which tracks a mirror list and exposes a static
>> > > > > > > > site
>> > > to
>> > > > be
>> > > > > > > able
>> > > > > > > > to
>> > > > > > > >     > query the mirror list.
>> > > > > > > >     >
>> > > > > > > >     > The web server will receive a request and will do a
>> > > > > > > > 302
>> > > > > redirect
>> > > > > > to
>> > > > > > > > the
>> > > > > > > >     > appropriate resource on one of the mirrors.
>> > > > > > > >     >
>> > > > > > > >     > This gets us started.  Legacy environments will have
>> > > > > > > > to do
>> > > a
>> > > > DB
>> > > > > > > > change to
>> > > > > > > >     > change from 'download.cloud.com' to '
>> > > > downloads.cloudstack.org'
>> > > > > > (or
>> > > > > > > >     > whatever), but otherwise it should be pretty simple.
>> > > > > > > >     >
>> > > > > > > >     > We can improve how we deliver templates going forward,
>> > > > > > > > but
>> > > > this
>> > > > > > > > obviously
>> > > > > > > >     > requires some discussion still.  I will try to buy us
>> > > > > > > > some
>> > > > time
>> > > > > > > with
>> > > > > > > > an
>> > > > > > > >     > implementation which solves our problems today...
>> > > > > > > >     >
>> > > > > > > >     > *Will STEVENS*
>> > > > > > > >     > Lead Developer
>> > > > > > > >     >
>> > > > > > > >     > <https://goo.gl/NYZ8KK>
>> > > > > > > >     >
>> > > > > > > >     > On Wed, Mar 1, 2017 at 7:52 AM, Abhinandan Prateek <
>> > > > > > > >     > abhinandan.prateek@shapeblue.com> wrote:
>> > > > > > > >     >
>> > > > > > > >     >>
>> > > > > > > >     >> Initial seeding is a manual step and that is not
>> > > > > > > > going to
>> > > > > change
>> > > > > > > in
>> > > > > > > > near
>> > > > > > > >     >> future. A handy list of official places from where
>> > > > > > > > these
>> > > > > > templates
>> > > > > > > > can be
>> > > > > > > >     >> downloaded won’t harm this part of the installation.
>> > > > > > > > Just
>> > > > > allow
>> > > > > > > > anyone
>> > > > > > > >     >> credible who follows democratic process to publish
>> > > > > > > > their
>> > > > > > templates
>> > > > > > > > with
>> > > > > > > >     >> apache’s blessings.
>> > > > > > > >     >>
>> > > > > > > >     >>
>> > > > > > > >     >> Coming to urls hard coded in db, the popular
>> > > > > > > > suggestion is
>> > > > to
>> > > > > > > point
>> > > > > > > > them
>> > > > > > > >     >> to mirrors. This looks good but this is going to take
>> > > > > > > > some
>> > > > > > effort
>> > > > > > > > that has
>> > > > > > > >     >> not materialised yet. Once this change is made it
>> > > > > > > > will be
>> > > > > going
>> > > > > > in
>> > > > > > > > some
>> > > > > > > >     >> future release like 4.11; the problem still remains
>> > > > > > > > for
>> > > the
>> > > > > > > > releases that
>> > > > > > > >     >> are already out there. In case access to
>> > > download.cloud.com
>> > > > > is
>> > > > > > > > dropped
>> > > > > > > >     >> at some point we will have to document the procedure
>> > > > > > > > such
>> > > > that
>> > > > > > > > people know
>> > > > > > > >     >> how to make these stable releases work. That is where
>> > > > > > > > the
>> > > > > > > procedure
>> > > > > > > >     >> documented here https://shankerbalan.net/blog/
>> > > > > > > >     >> seed-cloudstack-templates-offline/ should be
>> > > > > > > > officially
>> > > > > adapted
>> > > > > > > > with a
>> > > > > > > >     >> marketplace to pick up templates of choosing. Maybe
>> > > > > > > > we can
>> > > > > have
>> > > > > > a
>> > > > > > > > pre-setup
>> > > > > > > >     >> script to make choices instead of written wiki. Note
>> > > > > > > > that
>> > > > this
>> > > > > > > will
>> > > > > > > > be
>> > > > > > > >     >> anyway needed by the older releases.
>> > > > > > > >     >>
>> > > > > > > >     >>
>> > > > > > > >     >>
>> > > > > > > >     >>
>> > > > > > > >     >> On 01/03/17, 4:24 PM, "Paul Angus" <
>> > > > paul.angus@shapeblue.com>
>> > > > > > > > wrote:
>> > > > > > > >     >>
>> > > > > > > >     >> >-1
>> > > > > > > >     >> >
>> > > > > > > >     >> >We are trying to get MORE people to use CloudStack.
>> > > > > > > > And
>> > > > > > raising
>> > > > > > > > the
>> > > > > > > >     >> barrier to entry does nothing to help that.
>> > > > > > > >     >> >
>> > > > > > > >     >> >
>> > > > > > > >     >> >Kind regards,
>> > > > > > > >     >> >
>> > > > > > > >     >> >Paul Angus
>> > > > > > > >     >> >
>> > > > > > > >     >> >paul.angus@shapeblue.com
>> > > > > > > >     >> >www.shapeblue.com
>> > > > > > > >     >> >53 Chandos Place, Covent Garden, London  WC2N 4HSUK
>> > > > > > > >     >> >@shapeblue
>> > > > > > > >     >> >
>> > > > > > > >     >> >
>> > > > > > > >     >> >
>> > > > > > > >     >> >
>> > > > > > > >     >> >-----Original Message-----
>> > > > > > > >     >> >From: Abhinandan Prateek [mailto:abhinandan.prateek@
>> > > > > > > shapeblue.com]
>> > > > > > > >     >> >Sent: 01 March 2017 09:39
>> > > > > > > >     >> >To: dev@cloudstack.apache.org
>> > > > > > > >     >> >Subject: Re: Modern template hosting
>> > > > > > > >     >> >
>> > > > > > > >     >> >+1
>> > > > > > > >     >> >
>> > > > > > > >     >> >After seeing this discussion running in circles
>> > > > > > > > several
>> > > > > times,
>> > > > > > I
>> > > > > > > > think
>> > > > > > > >     >> we should at least get started with the simplest
>> option.
>> > > > > > > >     >> >
>> > > > > > > >     >> >
>> > > > > > > >     >> >
>> > > > > > > >     >> >
>> > > > > > > >     >> >On 27/02/17, 11:54 PM, "Chiradeep Vittal" <
>> > > > > > chiradeepv@gmail.com>
>> > > > > > > > wrote:
>> > > > > > > >     >> >
>> > > > > > > >     >> >>The alternative foolproof way is to simply not
>> > > > > > > > provide
>> > > the
>> > > > > > > > automatic
>> > > > > > > >     >> >>download.
>> > > > > > > >     >> >>
>> > > > > > > >     >> >>Just document the mirror list URL where the
>> > > > > > > > templates
>> > > can
>> > > > be
>> > > > > > > > found. I
>> > > > > > > >     >> >>don't know why people reject this option.
>> > > > > > > >     >> >>
>> > > > > > > >     >> >>https://shankerbalan.net/blog/seed-cloudstack-
>> > > > > > > templates-offline/
>> > > > > > > >     >> >>
>> > > > > > > >     >> >>On Mon, Feb 27, 2017 at 10:15 AM, Will Stevens
>> > > > > > > >     >> >><williamstevens@gmail.com>
>> > > > > > > >     >> >>wrote:
>> > > > > > > >     >> >>
>> > > > > > > >     >> >>> OK. Thanks for the heads up.
>> > > > > > > >     >> >>>
>> > > > > > > >     >> >>> On Feb 27, 2017 1:08 PM, "Chiradeep Vittal" <
>> > > > > > > > chiradeepv@gmail.com>
>> > > > > > > >     >> wrote:
>> > > > > > > >     >> >>>
>> > > > > > > >     >> >>> > Sounds workable. The downloader code in the
>> > > > > > > > SSVM
>> > > won't
>> > > > > > > follow
>> > > > > > > >     >> >>> > redirects I think.
>> > > > > > > >     >> >>> > https://github.com/apache/cloudstack/blob/
>> > > > > > > 5511065fc20787619d
>> > > > > > > > 9cd0444
>> > > > > > > >     >> >>> > a65a3 155fc9c921/core/src/com/cloud/
>> > > storage/template/
>> > > > > > > >     >> >>> > HttpTemplateDownloader.java#L93
>> > > > > > > >     >> >>> > https://goo.gl/dSi0r5
>> > > > > > > >     >> >>> >  Might need to add
>> > > > > > > >     >> >>> > client.setRedirectStrategy(new
>> > > LaxRedirectStrategy());
>> > > > > > > >     >> >>> >
>> > > > > > > >     >> >>> > On Mon, Feb 27, 2017 at 9:57 AM, Will Stevens
>> > > > > > > >     >> >>> > <wstevens@cloudops.com>
>> > > > > > > >     >> >>> > wrote:
>> > > > > > > >     >> >>> >
>> > > > > > > >     >> >>> > > We haven't opened a ticket yet because we
>> > > > > > > > don't
>> > > > have a
>> > > > > > > > strategy
>> > > > > > > >     >> yet.
>> > > > > > > >     >> >>> > >
>> > > > > > > >     >> >>> > > What do you guys think of this:
>> > > > > > > >     >> >>> > > - We setup a new github repo in the 'apache'
>> > > > > > > > org
>> > > > which
>> > > > > > > > consists
>> > > > > > > >     >> >>> > > of a
>> > > > > > > >     >> >>> > single
>> > > > > > > >     >> >>> > > file with a list of active/supported mirrors.
>> > > > > > > >     >> >>> > > - I write a small web server, distributed as
>> > > > > > > > a
>> > > > binary,
>> > > > > > > > which can
>> > > > > > > >     >> >>> > > be
>> > > > > > > >     >> >>> > hosted
>> > > > > > > >     >> >>> > > by ASF Infra.  This web server will query the
>> > > > current
>> > > > > > list
>> > > > > > > > of
>> > > > > > > >     >> >>> > > mirrors
>> > > > > > > >     >> >>> and
>> > > > > > > >     >> >>> > > will select one and then do a 302 redirect to
>> > > > > > > > that
>> > > > > > mirror.
>> > > > > > > >     >> >>> > >
>> > > > > > > >     >> >>> > > The act of 'choosing' a mirror could be done
>> > > > > > > > in a
>> > > > > number
>> > > > > > > of
>> > > > > > > > ways.
>> > > > > > > >     >> >>> > > - If we want to define an order, then it
>> > > > > > > > could
>> > > just
>> > > > > try
>> > > > > > > > from the
>> > > > > > > >     >> >>> > > top of
>> > > > > > > >     >> >>> > the
>> > > > > > > >     >> >>> > > list and work its way down.  It would curl
>> > > > > > > > the
>> > > > target
>> > > > > to
>> > > > > > > > make
>> > > > > > > >     >> >>> > > sure it
>> > > > > > > >     >> >>> > gets
>> > > > > > > >     >> >>> > > a 200 and if it does, it would do a 302
>> > redirect.
>> > > > > > > >     >> >>> > > - Or, if we want to distribute the load
>> > > > > > > > across the
>> > > > > > > mirrors,
>> > > > > > > > we
>> > > > > > > >     >> >>> > > could
>> > > > > > > >     >> >>> pick
>> > > > > > > >     >> >>> > > from the list randomly.  Again, doing a curl
>> > > > > > > > to
>> > > > verify
>> > > > > > the
>> > > > > > > > mirror
>> > > > > > > >     >> >>> > > is up
>> > > > > > > >     >> >>> > and
>> > > > > > > >     >> >>> > > then doing a redirect.
>> > > > > > > >     >> >>> > > - If we want to get fancy, we could do a
>> > > > > > > > reverse
>> > > IP
>> > > > > > lookup
>> > > > > > > > and
>> > > > > > > >     >> >>> > > try to
>> > > > > > > >     >> >>> > match
>> > > > > > > >     >> >>> > > the requester with their closest geographical
>> > > > mirror.
>> > > > > > > >     >> >>> > >
>> > > > > > > >     >> >>> > > Thoughts?
>> > > > > > > >     >> >>> > >
>> > > > > > > >     >> >>> > > *Will STEVENS*
>> > > > > > > >     >> >>> > > Lead Developer
>> > > > > > > >     >> >>> > >
>> > > > > > > >     >> >>> > > <https://goo.gl/NYZ8KK>
>> > > > > > > >     >> >>> > >
>> > > > > > > >     >> >>> > > On Mon, Feb 27, 2017 at 12:46 PM, Chiradeep
>> > > Vittal <
>> > > > > > > >     >> >>> chiradeepv@gmail.com
>> > > > > > > >     >> >>> > >
>> > > > > > > >     >> >>> > > wrote:
>> > > > > > > >     >> >>> > >
>> > > > > > > >     >> >>> > > > What steps are needed to set up a mirror?
>> > > > > > > > What
>> > > > does
>> > > > > > > Infra
>> > > > > > > > need
>> > > > > > > >     >> to do?
>> > > > > > > >     >> >>> > Has
>> > > > > > > >     >> >>> > > > anybody filed a ticket with Infra?
>> > > > > > > >     >> >>> > > >
>> > > > > > > >     >> >>> > > > On Sun, Feb 26, 2017 at 10:17 PM, Raja
>> > Pullela <
>> > > > > > > >     >> >>> > > > raja.pullela@accelerite.com>
>> > > > > > > >     >> >>> > > > wrote:
>> > > > > > > >     >> >>> > > >
>> > > > > > > >     >> >>> > > > > Hi will,
>> > > > > > > >     >> >>> > > > >
>> > > > > > > >     >> >>> > > > > I believe, we didn’t get to close
>> > > > > > > > ‘getting a
>> > > > > mirror
>> > > > > > on
>> > > > > > > > Apache’
>> > > > > > > >     >> >>> > because
>> > > > > > > >     >> >>> > > we
>> > > > > > > >     >> >>> > > > > needed someone on the Apache Infra side
>> > > > > > > > to
>> > > close
>> > > > > > this.
>> > > > > > > > BTW,
>> > > > > > > >     >> >>> > > > > cloudstack-apt.get.eu (I think Nux
>> > > > > > > > manages
>> > > > this?)
>> > > > > > has
>> > > > > > > >     >> >>> > > > > all/most of
>> > > > > > > >     >> >>> > the
>> > > > > > > >     >> >>> > > > > content.  Once we can close on the Apache
>> > > mirror
>> > > > > for
>> > > > > > > > hosting
>> > > > > > > >     >> >>> > > > > the
>> > > > > > > >     >> >>> > > > content, I
>> > > > > > > >     >> >>> > > > > can help assist getting the content there.
>> > > > > > > >     >> >>> > > > >
>> > > > > > > >     >> >>> > > > > For now, we have replicated the
>> > > > > download.cloud.com
>> > > > > > > > content to
>> > > > > > > >     >> >>> > > > > ‘ s3.download.accelerite.com’.
>> > > > > > > >     >> >>> > > > > Also, we are working on a set of
>> > > steps/procedure
>> > > > > to
>> > > > > > > > help with
>> > > > > > > >     >> >>> > > > > this change.  I will update everyone in
>> > > > > > > > about
>> > > a
>> > > > > > week’s
>> > > > > > > > time
>> > > > > > > >     >> >>> > > > > on the
>> > > > > > > >     >> >>> > details.
>> > > > > > > >     >> >>> > > > >
>> > > > > > > >     >> >>> > > > > Best,
>> > > > > > > >     >> >>> > > > > Raja Pullela
>> > > > > > > >     >> >>> > > > > Engineering Team,
>> > > > > > > >     >> >>> > > > > Accelerite, 2055 Laurelwood Road, Santa
>> > > > > > > > Clara,
>> > > > CA,
>> > > > > > > 95054
>> > > > > > > >     >> >>> > > > >
>> > > > > > > >     >> >>> > > > > On 2/24/17, 11:23 PM, "
>> > > williamstevens@gmail.com
>> > > > > on
>> > > > > > > > behalf of
>> > > > > > > >     >> >>> > > > > Will Stevens" <williamstevens@gmail.com
>> > > > > > > > on
>> > > > behalf
>> > > > > > of
>> > > > > > > >     >> >>> > wstevens@cloudops.com>
>> > > > > > > >     >> >>> > > > > wrote:
>> > > > > > > >     >> >>> > > > >
>> > > > > > > >     >> >>> > > > > unfortunately the template mirror
>> > > > > > > > conversation
>> > > > got
>> > > > > > > > caught up
>> > > > > > > >     >> >>> > > > > in
>> > > > > > > >     >> >>> > details
>> > > > > > > >     >> >>> > > > and
>> > > > > > > >     >> >>> > > > > nobody took the lead on implementing a
>> > > solution.
>> > > > > > > >     >> >>> > > > >
>> > > > > > > >     >> >>> > > > > citrix has been pinging me every couple
>> > > > > > > > months
>> > > > to
>> > > > > > say
>> > > > > > > > 'dude,
>> > > > > > > >     >> >>> > > > > we
>> > > > > > > >     >> >>> need
>> > > > > > > >     >> >>> > to
>> > > > > > > >     >> >>> > > > > remove the dependency on
>> > > > > > > > download.citrix.com
>> > > ',
>> > > > > but
>> > > > > > i
>> > > > > > > > have not
>> > > > > > > >     >> >>> > > > > had
>> > > > > > > >     >> >>> > the
>> > > > > > > >     >> >>> > > > > cycles to get in and solve the problem.
>> > > > > > > > the
>> > > > > > shutdown
>> > > > > > > > of that
>> > > > > > > >     >> >>> > > > > is
>> > > > > > > >     >> >>> > > imminent
>> > > > > > > >     >> >>> > > > > right now, so we need to solve it asap.
>> > > > > > > >     >> >>> > > > >
>> > > > > > > >     >> >>> > > > >
>> > > > > > > >     >> >>> > > > >
>> > > > > > > >     >> >>> > > > > *Will STEVENS*
>> > > > > > > >     >> >>> > > > > Lead Developer
>> > > > > > > >     >> >>> > > > >
>> > > > > > > >     >> >>> > > > > <https://goo.gl/NYZ8KK>
>> > > > > > > >     >> >>> > > > >
>> > > > > > > >     >> >>> > > > > On Fri, Feb 24, 2017 at 12:38 PM, Paul
>> > Angus <
>> > > > > > > >     >> >>> > paul.angus@shapeblue.com
>> > > > > > > >     >> >>> > > >
>> > > > > > > >     >> >>> > > > > wrote:
>> > > > > > > >     >> >>> > > > >
>> > > > > > > >     >> >>> > > > > > Hi Nathan,
>> > > > > > > >     >> >>> > > > > >
>> > > > > > > >     >> >>> > > > > > Ideally, if you put the template
>> > > > > > > > location in
>> > > > (or
>> > > > > > use
>> > > > > > > a
>> > > > > > > >     >> >>> > > > > > template
>> > > > > > > >     >> >>> > > defined
>> > > > > > > >     >> >>> > > > > > in)  test_data.py then the actual
>> > > > > > > > location
>> > > can
>> > > > > be
>> > > > > > > >     >> >>> > > > > > overridden by
>> > > > > > > >     >> >>> > > anyone
>> > > > > > > >     >> >>> > > > > > testing.
>> > > > > > > >     >> >>> > > > > >
>> > > > > > > >     >> >>> > > > > > For Trillian, we've copied all of the
>> > > > templates
>> > > > > > that
>> > > > > > > > people
>> > > > > > > >     >> >>> > > > > > have
>> > > > > > > >     >> >>> > > define
>> > > > > > > >     >> >>> > > > > to
>> > > > > > > >     >> >>> > > > > > a local repo and then replace the URLs
>> > > > > > > > in test_data.py to
>> > > > > > > >     >> >>> > > > > > reduce
>> > > > > > > >     >> >>> > > > > bandwidth
>> > > > > > > >     >> >>> > > > > > use and download times.
>> > > > > > > >     >> >>> > > > > >
>> > > > > > > >     >> >>> > > > > > Ie:
>> > > > > > > >     >> >>> > > > > >
>> > > > > > > >     >> >>> > > > > >             "bootableIso":
>> > > > > > > >     >> >>> > > > > >                 {
>> > > > > > > >     >> >>> > > > > >                     "displaytext": "Test
>> > > > > Bootable
>> > > > > > > > ISO",
>> > > > > > > >     >> >>> > > > > >                     "name": "testISO",
>> > > > > > > >     >> >>> > > > > >                     "bootable": True,
>> > > > > > > >     >> >>> > > > > >                     "ispublic": False,
>> > > > > > > >     >> >>> > > > > >                     "url": "{{
>> > > > > > > marvin_images_location
>> > > > > > > >     >> >>> > > > > > }}/TinyCore-current.iso",
>> > > > > > > >     >> >>> > > > > >                     "ostype": 'Other
>> Linux
>> > > > > > > (64-bit)',
>> > > > > > > >     >> >>> > > > > >                     "mode":
>> > 'HTTP_DOWNLOAD'
>> > > > > > > >     >> >>> > > > > >         },
>> > > > > > > >     >> >>> > > > > >
>> > > > > > > >     >> >>> > > > > >
>> > > > > > > >     >> >>> > > > > >
>> > > > > > > >     >> >>> > > > > > I thought that we had come up with a
>> > > solution
>> > > > > for
>> > > > > > > >     >> >>> > download.cloud.com
>> > > > > > > >     >> >>> > > ,
>> > > > > > > >     >> >>> > > > by
>> > > > > > > >     >> >>> > > > > > having a mirrorlist hosted in Community
>> > > Apache
>> > > > > > > > 'space' with
>> > > > > > > >     >> >>> anyone
>> > > > > > > >     >> >>> > > able
>> > > > > > > >     >> >>> > > > > to
>> > > > > > > >     >> >>> > > > > > out themselves forward as a mirror.
>> > > > > > > >     >> >>> > > > > > But I must admit I lost track of
>> > > > > > > > whether
>> > > > anyone
>> > > > > > made
>> > > > > > > > the
>> > > > > > > >     >> >>> requisite
>> > > > > > > >     >> >>> > > > > changes
>> > > > > > > >     >> >>> > > > > > in code....
>> > > > > > > >     >> >>> > > > > >
>> > > > > > > >     >> >>> > > > > >
>> > > > > > > >     >> >>> > > > > >
>> > > > > > > >     >> >>> > > > > >
>> > > > > > > >     >> >>> > > > > > Kind regards,
>> > > > > > > >     >> >>> > > > > >
>> > > > > > > >     >> >>> > > > > > Paul Angus
>> > > > > > > >     >> >>> > > > > >
>> > > > > > > >     >> >>> > > > > > paul.angus@shapeblue.com
>> > > > > > > >     >> >>> > > > > > www.shapeblue.com
>> > > > > > > >     >> >>> > > > > > 53 Chandos Place, Covent Garden, London
>> > > WC2N
>> > > > > > 4HSUK
>> > > > > > > >     >> >>> > > > > > @shapeblue
>> > > > > > > >     >> >>> > > > > >
>> > > > > > > >     >> >>> > > > > >
>> > > > > > > >     >> >>> > > > > >
>> > > > > > > >     >> >>> > > > > >
>> > > > > > > >     >> >>> > > > > > -----Original Message-----
>> > > > > > > >     >> >>> > > > > > From: williamstevens@gmail.com
>> > > > > > > >     >> >>> > > > > > [mailto:williamstevens@gmail.com]
>> > > > > > > >     >> >>> > On
>> > > > > > > >     >> >>> > > > > > Behalf Of Will Stevens
>> > > > > > > >     >> >>> > > > > > Sent: 24 February 2017 16:30
>> > > > > > > >     >> >>> > > > > > To: dev@cloudstack.apache.org
>> > > > > > > >     >> >>> > > > > > Subject: Re: Modern template hosting
>> > > > > > > >     >> >>> > > > > >
>> > > > > > > >     >> >>> > > > > > this is a hard questions.  in general,
>> > > > > > > > we
>> > > > should
>> > > > > > be
>> > > > > > > > setting
>> > > > > > > >     >> >>> > > > > > up a
>> > > > > > > >     >> >>> > > mirror
>> > > > > > > >     >> >>> > > > > on
>> > > > > > > >     >> >>> > > > > > some cloudstack/apache domain and then
>> > > mirror
>> > > > to
>> > > > > > > other
>> > > > > > > >     >> >>> > > > > > provided
>> > > > > > > >     >> >>> > > > > templates.
>> > > > > > > >     >> >>> > > > > >
>> > > > > > > >     >> >>> > > > > > we MUST come up with a solution to
>> > deprecate
>> > > > > > > >     >> >>> > > > > > 'download.cloud.com
>> > > > > > > >     >> >>> ',
>> > > > > > > >     >> >>> > > > that
>> > > > > > > >     >> >>> > > > > > is going to be going away any day now.
>> > > > > > > >     >> >>> > > > > >
>> > > > > > > >     >> >>> > > > > > i don't know the right way to solve
>> > > > > > > > this to
>> > > be
>> > > > > > > > honest, but
>> > > > > > > >     >> >>> > > > > > if you
>> > > > > > > >     >> >>> > > have
>> > > > > > > >     >> >>> > > > > > ideas, i am willing to help.
>> > > > > > > >     >> >>> > > > > >
>> > > > > > > >     >> >>> > > > > >
>> > > > > > > >     >> >>> > > > > >
>> > > > > > > >     >> >>> > > > > > *Will STEVENS*
>> > > > > > > >     >> >>> > > > > > Lead Developer
>> > > > > > > >     >> >>> > > > > >
>> > > > > > > >     >> >>> > > > > > <https://goo.gl/NYZ8KK>
>> > > > > > > >     >> >>> > > > > >
>> > > > > > > >     >> >>> > > > > > On Fri, Feb 24, 2017 at 11:25 AM,
>> > > > > > > > Nathan
>> > > > > Johnson <
>> > > > > > > >     >> >>> njohnson@ena.com
>> > > > > > > >     >> >>> > >
>> > > > > > > >     >> >>> > > > > wrote:
>> > > > > > > >     >> >>> > > > > >
>> > > > > > > >     >> >>> > > > > > > So not to re-open a can of worms, but
>> > > > > > > > I’m
>> > > > in a
>> > > > > > > > situation
>> > > > > > > >     >> >>> > > > > > > where
>> > > > > > > >     >> >>> I
>> > > > > > > >     >> >>> > > need
>> > > > > > > >     >> >>> > > > > > > to come up with a Marvin component
>> > > > > > > > test
>> > > that
>> > > > > > > > depends on a
>> > > > > > > >     >> >>> > template
>> > > > > > > >     >> >>> > > > > > > based on a kernel that’s relatively
>> > > > > > > > new,
>> > > > i.e.,
>> > > > > > > > newer than
>> > > > > > > >     >> >>> Centos
>> > > > > > > >     >> >>> > > 5.3
>> > > > > > > >     >> >>> > > > /
>> > > > > > > >     >> >>> > > > > > Ubuntu 10.04 .
>> > > > > > > >     >> >>> > > > > > > I see openvm.eu has a suitable
>> > > > > > > > template
>> > > > > (Ubuntu
>> > > > > > > > 16.0.4
>> > > > > > > >     >> >>> > > > > > > for
>> > > > > > > >     >> >>> KVM),
>> > > > > > > >     >> >>> > > but
>> > > > > > > >     >> >>> > > > > > > from looking at the thread "Migrating
>> > > > > CloudStack
>> > > > > > > > content
>> > > > > > > >     >> >>> > > > > > > from download.cloud.com” it looks
>> > > > > > > > like
>> > > > there
>> > > > > is
>> > > > > > > >     >> >>> > > > > > > resistance to using
>> > > > > > > >     >> >>> > > this
>> > > > > > > >     >> >>> > > > at
>> > > > > > > >     >> >>> > > > > > > least for hosting system vm templates
>> > > > > > > > over
>> > > > > > > concerns
>> > > > > > > > of
>> > > > > > > >     >> >>> > neutrality.
>> > > > > > > >     >> >>> > > > > > > Would this be suitable for a
>> > > > > > > > component
>> > > test?
>> > > > > If
>> > > > > > > > not,
>> > > > > > > >     >> >>> > > > > > > what is a
>> > > > > > > >     >> >>> > > > > “blessed”
>> > > > > > > >     >> >>> > > > > > > template location?
>> > > > > > > >     >> >>> > > > > > >
>> > > > > > > >     >> >>> > > > > > > Thanks in advance!
>> > > > > > > >     >> >>> > > > > > >
>> > > > > > > >     >> >>> > > > > > > Nathan Johnson
>> > > > > > > >     >> >>> > > > > > > R&D Engineer
>> > > > > > > >     >> >>> > > > > > >
>> > > > > > > >     >> >>> > > > > > >
>> > > > > > > >     >> >>> > > > > > >
>> > > > > > > >     >> >>> > > > > > > 618 Grassmere Park Drive, Suite 12
>> > > > Nashville,
>> > > > > TN
>> > > > > > > > 37211
>> > > > > > > >     >> >>> > > > > > > General Office: 615-312-6000
>> > > > > > > >     >> >>> > > > > > >
>> > > > > > > >     >> >>> > > > > > > website | blog | support
>> > > > > > > >     >> >>> > > > > > >
>> > > > > > > >     >> >>> > > > > > >
>> > > > > > > >     >> >>> > > > > > >
>> > > > > > > >     >> >>> > > > > > >
>> > > > > > > >     >> >>> > > > > > >
>> > > > > > > >     >> >>> > > > > > >
>> > > > > > > >     >> >>> > > > > > >
>> > > > > > > >     >> >>> > > > > >
>> > > > > > > >     >> >>> > > > >
>> > > > > > > >     >> >>> > > > >
>> > > > > > > >     >> >>> > > > >
>> > > > > > > >     >> >>> > > > >
>> > > > > > > >     >> >>> > > > >
>> > > > > > > >     >> >>> > > > >
>> > > > > > > >     >> >>> > > > > DISCLAIMER
>> > > > > > > >     >> >>> > > > > ==========
>> > > > > > > >     >> >>> > > > > This e-mail may contain privileged and
>> > > > > confidential
>> > > > > > > >     >> >>> > > > > information
>> > > > > > > >     >> >>> which
>> > > > > > > >     >> >>> > > is
>> > > > > > > >     >> >>> > > > > the property of Accelerite, a Persistent
>> > > Systems
>> > > > > > > > business. It
>> > > > > > > >     >> >>> > > > > is
>> > > > > > > >     >> >>> > > intended
>> > > > > > > >     >> >>> > > > > only for the use of the individual or
>> > > > > > > > entity
>> > > to
>> > > > > > which
>> > > > > > > > it is
>> > > > > > > >     >> >>> > addressed.
>> > > > > > > >     >> >>> > > If
>> > > > > > > >     >> >>> > > > > you are not the intended recipient, you
>> > > > > > > > are
>> > > not
>> > > > > > > > authorized to
>> > > > > > > >     >> >>> > > > > read,
>> > > > > > > >     >> >>> > > > retain,
>> > > > > > > >     >> >>> > > > > copy, print, distribute or use this
>> message.
>> > > If
>> > > > > you
>> > > > > > > have
>> > > > > > > >     >> >>> > > > > received
>> > > > > > > >     >> >>> > this
>> > > > > > > >     >> >>> > > > > communication in error, please notify the
>> > > sender
>> > > > > and
>> > > > > > > > delete
>> > > > > > > >     >> >>> > > > > all
>> > > > > > > >     >> >>> > copies
>> > > > > > > >     >> >>> > > of
>> > > > > > > >     >> >>> > > > > this message. Accelerite, a Persistent
>> > > > > > > > Systems
>> > > > > > > business
>> > > > > > > > does
>> > > > > > > >     >> >>> > > > > not
>> > > > > > > >     >> >>> > accept
>> > > > > > > >     >> >>> > > > any
>> > > > > > > >     >> >>> > > > > liability for virus infected mails.
>> > > > > > > >     >> >>> > > > >
>> > > > > > > >     >> >>> > > >
>> > > > > > > >     >> >>> > >
>> > > > > > > >     >> >>> >
>> > > > > > > >     >> >>>
>> > > > > > > >     >> >
>> > > > > > > >     >> >abhinandan.prateek@shapeblue.com
>> > > > > > > >     >> >www.shapeblue.com
>> > > > > > > >     >> >53 Chandos Place, Covent Garden, London  WC2N 4HSUK
>> > > > > @shapeblue
>> > > > > > > >     >> >
>> > > > > > > >     >> >
>> > > > > > > >     >> >
>> > > > > > > >     >>
>> > > > > > > >     >> abhinandan.prateek@shapeblue.com
>> > > > > > > >     >> www.shapeblue.com
>> > > > > > > >     >> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
>> > > > > > > >     >> @shapeblue
>> > > > > > > >     >>
>> > > > > > > >     >>
>> > > > > > > >     >>
>> > > > > > > >     >>
>> > > > > > > >     >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > daan.hoogland@shapeblue.com
>> > > > > > > > www.shapeblue.com
>> > > > > > > > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
>> > > > > > > > @shapeblue
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > --
>> > > > > > > Rafael Weingärtner
>> > > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > --
>> > > > > > Rafael Weingärtner
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>


Mime
View raw message