incubator-ooo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dennis E. Hamilton" <orc...@apache.org>
Subject RE: [DL LOGIC] How to choose a mirror when more than 1 is available?
Date Sat, 07 Apr 2012 23:20:50 GMT
>From the original post, I see these statements:

>> To get this all under one hat, we have to improve the download logic
>> that is currently done by JavaScript.
>>
>> To fulfill the first condition the download requests have to be split-up
>> to more than 1 central mirror redirector.

I assume that you want a script that will choose among multiple mirror
redirectors, when available.

Although it is not reflected directly, it appears that the idea is that 
the download requests be distributed in a weighted manner among available 
redirectors.  The allocation (as percentage of requests a redirector will 
receive over time) is presumably intended to deal with the portion of peak 
demand that a particular redirector has offered to handle. 

The commitment is to not exceed the expected proportion on a sustained
basis while providing a given redirector that that proportion of the 
requests over time (being potentially important for add-generated revenue
of the redirector systems).

When one of the redirectors is unavailable, there is a default redirector 
that will absorb all additional requests.  The allocation to non-default
redirectors is not increased.

NOTES

 1. This does not detect when peak load exceeds the agreed capacity 
    limit of any redirector system.

 2. This does not provide any mitigation when a redirector is unable 
    to service a request forwarded to it.

 3. Allocating requests is not the same as allocating load.  When
    requests are failing, the rapid resubmission of unaccepted requests
    can become a problem.

SAMPLE APPROACH

 1. The redirector services are named by simple strings.  

 2. The load management is specified in a single array, 

    var redirectorLoading = [ [factor-0, name-0], 
                              [factor-1, name-1], 
                              ...,
                              [factor-n, name-n]];

    where factor-i is a number from 0 to 100 specifying the 
    percentage proportion of requests that the redirector 
    service identified by name-i will accept.

    For the last entry, factor-n=100 and name-n is the string 
    name of the backup redirector that will absorb all
    unallocated requests.  This can be the same as a name-i 
    already in the mix.

    The special name "noservice" is used when there is no
    available service for the request.  
                
    Maintenance: To make a service unavailable, simply set its 
    proportion, factor-i, to 0.
    If the backup is unavailable, replace name-n by the name
    of an alternative backup, if any.  If there is no 
    alternative to the backup, change factor-n to 0 or 
    name-n to "noservice".

    I assume that this declaration would be in a place that is 
    easy to maintain without risking errors in maintenance of 
    anything else.

 3. One way to do the allocation is with some sort of
    simple function along these lines:

    function chooseService( )
    {  var nService = redirectorLoading.length;
       if (!nService) return "noservice";

       var nLottery = Math.floor(Math.random()*100);
       for (var i=0; i<nService; i++)
         { var loading = redirectorLoading[i][0];
           if (!loading && nLottery < loading)
               return redirectorLoading[i][1];
           nLottery -= loading;
           }
       return "noservice";
       }

-----Original Message-----
From: Marcus (OOo) [mailto:marcus.mail@wtnet.de] 
Sent: Saturday, April 07, 2012 12:08
To: ooo-dev@incubator.apache.org
Subject: Re: [DL LOGIC] How to choose a mirror when more than 1 is available?

Am 04/07/2012 07:49 PM, schrieb Dennis E. Hamilton:
> Comments in-line.
>
> I think it would be good to step back and agree on what problem that is being solved
and how should failure cases and intervention work.  Then it can be determined whether candidate
code addresses the problem or not.
>
>   - Dennis

OK, do you have any suggestions different to my initial post in this thread?

> -----Original Message-----
> From: Marcus (OOo) [mailto:marcus.mail@wtnet.de]
> Sent: Saturday, April 07, 2012 10:13
> To: ooo-dev@incubator.apache.org
> Subject: Re: [DL LOGIC] How to choose a mirror when more than 1 is available?
>
> [ ... ]
>
>>    2. The requests are distributed randomly based on the allocation
>   >    percentages to the systems whether or not they are available.
>
> Yes, but only when they are available. If not they get no requests.
>
>>    3. For any mirror system that receives a turn when it is not
>   >    available, the backup system is used.
>
> The non-available system won't get a request. It's directly forwarded to
> another one.
>
> <orcnote>
> The current logic does not try a different one if the random choice is in-range for a
non-available mirror system.  It always goes to the backup.  There is no retry.  You can see
that by running the sample page a couple of times.
> </orcnote>
>
> [ ... ]

Yes, thats correct.

>> My only other observations are
>>
>>    a. It should be easier to identify the systems and their
>   >    allocations and to update that information in the script.
>
> All values are (or should be) in the "globalvars.js" file. So, only in
> one location there are changes necessary.
>
> <orcnote>
>    globalvars.js is not a one-stop shopping place for adjusting the allocations and availabilities
with a simple change.  It's a tremendous warehouse of global settings of all kinds.
>    globalvars.js is a dangerous thing to touch in order to change allocations and/or
workaround mirror unavailability.
> </orcnote>

Where else do you see important values that need to be changed?
Why is it dangerous? Every web application has a kind of property 
management. Just change the value here instead of delivering changed 
code on many different places.

>>    b. If availability is to be determined dynamically, it is
>   >    probably better to check availability only after determining
>   >    that the system has been selected by the random procedure.
>   >    Also, if availability is configured manually, there is probably
>   >    some shared dataset that should hold the availability information
>   >    (and the allocations?) so the script does not have to be touched
>   >    and outages/re-allocations can be done quickly and reliably.
>
> No, it's not done dynamically (yet). We have to know if a system is
> down. Then we can disable it in the "globalvars.js" file.
>
> BTW:
> Sorry, if the quoted text is now messed-up. It was extremly difficult to
> answer as there was no line wrapping.

Marcus



>> -----Original Message-----
>> From: Marcus (OOo) [mailto:marcus.mail@wtnet.de]
>> Sent: Saturday, April 07, 2012 07:13
>> To: ooo-dev@incubator.apache.org
>> Subject: [DL LOGIC] How to choose a mirror when more than 1 is available?
[ ... ]
>> To get this all under one hat, we have to improve the download logic
>> that is currently done by JavaScript.
>>
>> To fulfill the first condition the download requests have to be split-up
>> to more than 1 central mirror redirector.
[ ... ]


Mime
View raw message