incubator-ooo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcus (OOo)" <marcus.m...@wtnet.de>
Subject Re: [DL LOGIC] How to choose a mirror when more than 1 is available?
Date Sun, 08 Apr 2012 02:20:26 GMT
Am 04/08/2012 01:20 AM, schrieb Dennis E. Hamilton:
>  From the original post, I see these statements:
>
>>> To get this all under one hat, we have to improve the download logic
>>> that is currently done by JavaScript.
>>>
>>> To fulfill the first condition the download requests have to be split-up
>>> to more than 1 central mirror redirector.
>
> I assume that you want a script that will choose among multiple mirror
> redirectors, when available.

Yes

> Although it is not reflected directly, it appears that the idea is that
> the download requests be distributed in a weighted manner among available
> redirectors.  The allocation (as percentage of requests a redirector will
> receive over time) is presumably intended to deal with the portion of peak
> demand that a particular redirector has offered to handle.

Yes

> The commitment is to not exceed the expected proportion on a sustained
> basis while providing a given redirector that that proportion of the
> requests over time (being potentially important for add-generated revenue
> of the redirector systems).
>
> When one of the redirectors is unavailable, there is a default redirector
> that will absorb all additional requests.  The allocation to non-default
> redirectors is not increased.

The default will be the next powerful redirector that is given by the 
weighted values.

> NOTES
>
>   1. This does not detect when peak load exceeds the agreed capacity
>      limit of any redirector system.

Maybe. But I think we can deal with this later if becomes a problem.
I wanted to have a simple script for the start.

>   2. This does not provide any mitigation when a redirector is unable
>      to service a request forwarded to it.

This should not happen. Except the redirector is down in the second of 
the download request.

I've no idea how to recognize automatically when a redirector is down. 
Except to ping it al lthe time. But this cannot be the solution.

>   3. Allocating requests is not the same as allocating load.  When
>      requests are failing, the rapid resubmission of unaccepted requests
>      can become a problem.

Right.

Sorry, but I think the rest is a bit to much for me. I'm not a usual 
developer and don't really understand it.

> SAMPLE APPROACH
>
>   1. The redirector services are named by simple strings.
>
>   2. The load management is specified in a single array,
>
>      var redirectorLoading = [ [factor-0, name-0],
>                                [factor-1, name-1],
>                                ...,
>                                [factor-n, name-n]];
>
>      where factor-i is a number from 0 to 100 specifying the
>      percentage proportion of requests that the redirector
>      service identified by name-i will accept.
>
>      For the last entry, factor-n=100 and name-n is the string
>      name of the backup redirector that will absorb all
>      unallocated requests.  This can be the same as a name-i
>      already in the mix.
>
>      The special name "noservice" is used when there is no
>      available service for the request.
>
>      Maintenance: To make a service unavailable, simply set its
>      proportion, factor-i, to 0.

I think that's the same as in my script example to set a mirror host to 0.

>      If the backup is unavailable, replace name-n by the name
>      of an alternative backup, if any.  If there is no
>      alternative to the backup, change factor-n to 0 or
>      name-n to "noservice".
>
>      I assume that this declaration would be in a place that is
>      easy to maintain without risking errors in maintenance of
>      anything else.
>
>   3. One way to do the allocation is with some sort of
>      simple function along these lines:
>
>      function chooseService( )
>      {  var nService = redirectorLoading.length;
>         if (!nService) return "noservice";
>
>         var nLottery = Math.floor(Math.random()*100);
>         for (var i=0; i<nService; i++)
>           { var loading = redirectorLoading[i][0];
>             if (!loading&&  nLottery<  loading)
>                 return redirectorLoading[i][1];
>             nLottery -= loading;
>             }
>         return "noservice";
>         }

Marcus



> -----Original Message-----
> From: Marcus (OOo) [mailto:marcus.mail@wtnet.de]
> Sent: Saturday, April 07, 2012 12:08
> To: ooo-dev@incubator.apache.org
> Subject: Re: [DL LOGIC] How to choose a mirror when more than 1 is available?
>
> Am 04/07/2012 07:49 PM, schrieb Dennis E. Hamilton:
>> Comments in-line.
>>
>> I think it would be good to step back and agree on what problem that is being solved
and how should failure cases and intervention work.  Then it can be determined whether candidate
code addresses the problem or not.
>>
>>    - Dennis
>
> OK, do you have any suggestions different to my initial post in this thread?
>
>> -----Original Message-----
>> From: Marcus (OOo) [mailto:marcus.mail@wtnet.de]
>> Sent: Saturday, April 07, 2012 10:13
>> To: ooo-dev@incubator.apache.org
>> Subject: Re: [DL LOGIC] How to choose a mirror when more than 1 is available?
>>
>> [ ... ]
>>
>>>     2. The requests are distributed randomly based on the allocation
>>    >     percentages to the systems whether or not they are available.
>>
>> Yes, but only when they are available. If not they get no requests.
>>
>>>     3. For any mirror system that receives a turn when it is not
>>    >     available, the backup system is used.
>>
>> The non-available system won't get a request. It's directly forwarded to
>> another one.
>>
>> <orcnote>
>> The current logic does not try a different one if the random choice is in-range for
a non-available mirror system.  It always goes to the backup.  There is no retry.  You can
see that by running the sample page a couple of times.
>> </orcnote>
>>
>> [ ... ]
>
> Yes, thats correct.
>
>>> My only other observations are
>>>
>>>     a. It should be easier to identify the systems and their
>>    >     allocations and to update that information in the script.
>>
>> All values are (or should be) in the "globalvars.js" file. So, only in
>> one location there are changes necessary.
>>
>> <orcnote>
>>     globalvars.js is not a one-stop shopping place for adjusting the allocations
and availabilities with a simple change.  It's a tremendous warehouse of global settings of
all kinds.
>>     globalvars.js is a dangerous thing to touch in order to change allocations and/or
workaround mirror unavailability.
>> </orcnote>
>
> Where else do you see important values that need to be changed?
> Why is it dangerous? Every web application has a kind of property
> management. Just change the value here instead of delivering changed
> code on many different places.
>
>>>     b. If availability is to be determined dynamically, it is
>>    >     probably better to check availability only after determining
>>    >     that the system has been selected by the random procedure.
>>    >     Also, if availability is configured manually, there is probably
>>    >     some shared dataset that should hold the availability information
>>    >     (and the allocations?) so the script does not have to be touched
>>    >     and outages/re-allocations can be done quickly and reliably.
>>
>> No, it's not done dynamically (yet). We have to know if a system is
>> down. Then we can disable it in the "globalvars.js" file.
>>
>> BTW:
>> Sorry, if the quoted text is now messed-up. It was extremly difficult to
>> answer as there was no line wrapping.
>
> Marcus
>
>
>
>>> -----Original Message-----
>>> From: Marcus (OOo) [mailto:marcus.mail@wtnet.de]
>>> Sent: Saturday, April 07, 2012 07:13
>>> To: ooo-dev@incubator.apache.org
>>> Subject: [DL LOGIC] How to choose a mirror when more than 1 is available?
> [ ... ]
>>> To get this all under one hat, we have to improve the download logic
>>> that is currently done by JavaScript.
>>>
>>> To fulfill the first condition the download requests have to be split-up
>>> to more than 1 central mirror redirector.
> [ ... ]

Mime
View raw message