incubator-ooo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcus (OOo)" <>
Subject Re: [DL LOGIC] How to choose a mirror when more than 1 is available?
Date Sat, 07 Apr 2012 17:13:03 GMT
Am 04/07/2012 05:41 PM, schrieb Dennis E. Hamilton:
> I looked at your JavaScript and it appears to me that what you
 > propose is as follows:
>   1. There is a fixed %-allocation among a number of available
 >   mirror systems (2 and 3 being illustrated with different
 >   procedures) plus a designated backup (which may be in the list).

Yes, that was Rob's first thought and I've scripted around it as I had 
no better idea for the moment.

If a mirror system is down (which can happen more often than thought, 
(e.g., for a short service work) a backup system is needed.

>   2. The requests are distributed randomly based on the allocation
 >   percentages to the systems whether or not they are available.

Yes, but only when they are available. If not they get no requests.

>   3. For any mirror system that receives a turn when it is not
 >   available, the backup system is used.

The non-available system won't get a request. It's directly forwarded to 
another one.

> In this approach, the portion of the requests that any available
 > mirror system but the backup receives does not change.  If the
 > randomly-chosen mirror is unavailable and the backup is also
 > unavailable, the request will not be satisfied whether or not
 > there are other mirror systems that are still available.

When both are down then it means that Apache and SourceForge are down at 
the same time. If this will happen we have really a problem. That's also 
an argument to get MirrorBrain into the game as third mirror host.

The time that this could happen should be really rare. And even when, we 
could show a temporary download page with static links to mirrors.

 > (Anyone who suspects this will get in by repeating requests until a
 > winner is found, which will overcome the allocation of load anyhow
 > and/or annoy users. It will be important how the failure to find a
 > mirror system is described.  And it will still be difficult to
 > trouble-shoot reports of failures on the lists and forums.)

We can show somewhere the mirror host as source.

Furthermore I expect to have this feature as One-Donwload-Click) on the 
"index.html". I expect that there will still a "other.html" with a table 
of available builds. Here we can point the user to.

> My only other observations are
>   a. It should be easier to identify the systems and their
 >   allocations and to update that information in the script.

All values are (or should be) in the "globalvars.js" file. So, only in 
one location there are changes necessary.

>   b. If availability is to be determined dynamically, it is
 >   probably better to check availability only after determining
 >   that the system has been selected by the random procedure.
 >   Also, if availability is configured manually, there is probably
 >   some shared dataset that should hold the availability information
 >   (and the allocations?) so the script does not have to be touched
 >   and outages/re-allocations can be done quickly and reliably.

No, it's not done dynamically (yet). We have to know if a system is 
down. Then we can disable it in the "globalvars.js" file.

Sorry, if the quoted text is now messed-up. It was extremly difficult to 
answer as there was no line wrapping.


> -----Original Message-----
> From: Marcus (OOo) []
> Sent: Saturday, April 07, 2012 07:13
> To:
> Subject: [DL LOGIC] How to choose a mirror when more than 1 is available?
> For our new release we will have a changed set of conditions:
> - new central mirror hosts for redirecting download requests
> - therefore also a new set of mirror server that host our builds
> - new set of supported platforms and languages
> - slighty changed file name schema
> To get this all under one hat, we have to improve the download logic
> that is currently done by JavaScript.
> To fulfill the first condition the download requests have to be split-up
> to more than 1 central mirror redirector.
> Here is a suggestion for choosing when (by random number) to redirect to
> which mirror:
> Test results:
> - to make debugging easier I've included some screen output and comments
> - I've tested it locally with different combinations of active/inactive
>     hosts and it works well
> - the fallback method works well, too
> - I've tested different percent values as ratio, but not yet for the
>     2nd case
> What do you think?
> Marcus

View raw message