incubator-ooo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Schaefer <joe_schae...@yahoo.com>
Subject Re: [DL LOGIC] How to choose a mirror when more than 1 is available?
Date Sun, 08 Apr 2012 03:55:03 GMT
Simplest thing to do is visit the opennlp example and use the CMS 

to view source.  The opennlp "ezt template file" is just a download.mdtext
filewith special braced text (that's NOT valid markdown) that ezmlm
interprets at script runtime and substitutes in values from the script.
There is some basic flow control in ezt with if/then and foreach loops etc
similar to what django templates do but with less syntactic fluff.

FYI ezt.py is a python template lib that Greg Stein wrote and is used throughout
Apache Infra.  The public url to the underlying cgi script is at 

http://www.apache.org/dyn/mirrors/mirrors.cgi

again the CMS will take you directly to the source code.


----- Original Message -----
> From: Dave Fisher <dave2wave@comcast.net>
> To: ooo-dev@incubator.apache.org
> Cc: 
> Sent: Saturday, April 7, 2012 11:01 PM
> Subject: Re: [DL LOGIC] How to choose a mirror when more than 1 is available?
> 
> Hi Joe,
> 
> While everyone else might be ignoring the distinction between the Apache closer 
> cgi/ezt and the current OOo javascript methods, I haven't missed the 
> difference. One is server and the other client.
> 
> You've mentioned where the various example pieces are, but to make it really 
> easy would you mind providing the two or three key urls into the svn repos? 
> I'm interested, but not in a hurry.
> 
> Thanks,
> Dave
> 
> On Apr 7, 2012, at 1:09 PM, Joe Schaefer wrote:
> 
>>  If you're really paranoid about the ASF mirrors
>>  you can use ezt to embed the [preferred] mirror
>>  inside some javascript within the page and run
>>  a test HEAD request on the url to ensure it exists.
>> 
>>  I don't see the point of the MirrorBrain stuff 
>> 
>>  at this point tho- will just make things more messy
>>  than they already are dealing with SF + ASF mirrors.
>> 
>> 
>> 
>>  ----- Original Message -----
>>>  From: Rob Weir <robweir@apache.org>
>>>  To: ooo-dev@incubator.apache.org
>>>  Cc: 
>>>  Sent: Saturday, April 7, 2012 4:03 PM
>>>  Subject: Re: [DL LOGIC] How to choose a mirror when more than 1 is 
> available?
>>> 
>>>  On Sat, Apr 7, 2012 at 3:35 PM, Marcus (OOo) 
> <marcus.mail@wtnet.de> wrote:
>>>>  Am 04/07/2012 07:38 PM, schrieb Marcus (OOo):
>>>> 
>>>>>  Am 04/07/2012 05:34 PM, schrieb Rob Weir:
>>>>>> 
>>>>>>  On Sat, Apr 7, 2012 at 10:13 AM, Marcus 
>>>  (OOo)<marcus.mail@wtnet.de>
>>>>>>  wrote:
>>>>>>> 
>>>>>>>  For our new release we will have a changed set of 
> conditions:
>>>>>>> 
>>>>>>>  - new central mirror hosts for redirecting download 
> requests
>>>>>>>  - therefore also a new set of mirror server that host 
> our 
>>>  builds
>>>>>>>  - new set of supported platforms and languages
>>>>>>>  - slighty changed file name schema
>>>>>>> 
>>>>>>>  To get this all under one hat, we have to improve the 
> download 
>>>  logic
>>>>>>>  that is
>>>>>>>  currently done by JavaScript.
>>>>>>> 
>>>>>>>  To fulfill the first condition the download requests 
> have to be
>>>>>>>  split-up to
>>>>>>>  more than 1 central mirror redirector.
>>>>>>> 
>>>>>>>  Here is a suggestion for choosing when (by random 
> number) to 
>>>  redirect to
>>>>>>>  which mirror:
>>>>>>> 
>>>>>>> 
> http://www.openoffice.org/download/test/index_new_dl.html
>>>>>>> 
>>>>>>>  Test results:
>>>>>>>  - to make debugging easier I've included some 
> screen output 
>>>  and comments
>>>>>>>  - I've tested it locally with different 
> combinations of 
>>>  active/inactive
>>>>>>>  hosts and it works well
>>>>>>>  - the fallback method works well, too
>>>>>>>  - I've tested different percent values as ratio, 
> but not 
>>>  yet for the
>>>>>>>  2nd case
>>>>>>> 
>>>>>>>  What do you think?
>>>>>>> 
>>>>>> 
>>>>>>  The logic on the test pages works for me.
>>>>>> 
>>>>>>  But two additional complexities:
>>>>>> 
>>>>>>  1) MirrorBrain let's us link directly to a download 
> file. We 
>>>  then
>>>>>>  send the user to 
> http://www.openoffice.org/download/contribute.html
>>>>>>  while the file is downloading. But SourceForge and Apache 
> mirror
>>>>>>  systems take the user to a different page (not controlled 
> by us). 
>>>  So
>>>>>>  in those cases the user does not see the contribute.html 
> page.
>>>>> 
>>>>> 
>>>>>  This should work also for SourceForge. The root path is always 
> the 
>>>  same.
>>>>>  We just need to add a subdir and file name.
>>>>> 
>>>>>  I don't know how to do it for Apache.
>>>>> 
>>>>>>  Since the contribute.html page has very useful information 
> for the
>>>>>>  user, like links to the install instructions, information 
> on 
>>>  support,
>>>>>>  contributing to the project, etc., I think it is important 
> that the
>>>>>>  user sees this information in all paths.
>>>>> 
>>>>> 
>>>>>  ACK
>>>>> 
>>>>>>  What could we do? Pop-up (or pop-under) a new HTML page? 
> But that 
>>>  is
>>>>>>  annoying to some users.
>>>>> 
>>>>> 
>>>>>  Right
>>>>> 
>>>>>>  The Apache and SF approaches both direct a user off of the
>>>>>>  OpenOffice.org website, which is risky.
>>>>>> 
>>>>>>  2) What if we have more than three mirror systems? It 
> sounds like 
>>>  you
>>>>>>  prefer to keep MirrorBrain. That is fine with me. So maybe 
> we have
>>>>>>  Apache, SF and MirrorBrain?
>>>>> 
>>>>> 
>>>>>  Yes, would be great. Another available system as possible 
> backup is not
>>>>>  bad.
>>>>> 
>>>>>  For any change of the number of available systems we have to 
> adapt the
>>>>>  JS logic.
>>>>> 
>>>>>>  Imagine an array of mirror networks, each with weights, in 
> sorted
>>>>>>  order from smallest to largest. (or sort the array in code)
>>>>>> 
>>>>>>  Apache:1
>>>>>>  MirrorBrain:2
>>>>>>  SourceForge:3
>>>>>> 
>>>>>>  TotalWeight = MirrorBrain + Apache + SourceForge == 6
>>>>>> 
>>>>>>  x = rand()
>>>>>> 
>>>>>>  if (x< Apache/TotalWeight)
>>>>>>  doApache()
>>>>>>  else if (x< MirrorBrain/TotalWeight)
>>>>>>  doMirrorBrain()
>>>>>>  else
>>>>>>  doSourceForge()
>>>>>> 
>>>>>>  Something like that.
>>>>> 
>>>>> 
>>>>>  Redirecting due to weighted values looks good. But also this is 
> working
>>>>>  with a fixed set of mirror hosts. A forth had to be build in 
> and
>>>>>  therefore the logic has to be adapted.
>>>>> 
>>>>>  Thanks for your feedback. I'll adapt the script later in 
> the 
>>>  evening.
>>>> 
>>>> 
>>>>  I've adapted it now. But I'm not sure if it's working 
> correct 
>>>  with some
>>>>  disabled mirrors.
>>>> 
>>> 
>>>  I'm not sure my logic was right in the above code.  For example, if
>>>  two mirrors have the same weight it would always pick the first one in
>>>  the array.
>>> 
>>>  But take a look at the approach in this article:
>>> 
>>>  http://www.javascriptkit.com/javatutors/weighrandom2.shtml
>>> 
>>>  I think that is much clearer and easier to use.
>>> 
>>>  Essentially, it says if you want weighted random selection with weights 
> like:
>>> 
>>>  Apache:1
>>>  MirrorBrain:2
>>>  SourceForge:3
>>> 
>>>  Then create an array with each item repeated according to its weight, 
> so:
>>> 
>>>  Apache
>>>  MirrorBrain
>>>  MirrorBrain
>>>  SourceForge
>>>  SourceForge
>>>  SourceForge
>>> 
>>>  Then just pick a random number 0...length-1 and return the value at 
> that index.
>>> 
>>>  As for unavailable mirrors, or even unavailable servers within a
>>>  mirror network, is there really anything we can do?  Other than tell
>>>  the user to refresh and try again?
>>> 
>>>  -Rob
>>> 
>>>>  Marcus
>>> 
> 

Mime
View raw message