Mailing-List: contact ooo-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: ooo-dev@incubator.apache.org
Received-SPF: pass (athena.apache.org: domain of jogischmidt@googlemail.com
 designates 209.85.214.47 as permitted sender)
Message-ID: <4EE0A9F2.4070300@googlemail.com>
Date: Thu, 08 Dec 2011 13:13:38 +0100
From: =?UTF-8?B?SsO8cmdlbiBTY2htaWR0?= <jogischmidt@googlemail.com>
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7;
 rv:8.0) Gecko/20111105 Thunderbird/8.0
MIME-Version: 1.0
To: ooo-dev@incubator.apache.org
Subject: Re: Extensions and templates
References: 
 <CAP-ksohR7w-=1v9-D=yWjNuebURp94Txm1iMjinUamXCPY0Qjw@mail.gmail.com>
 <4EDFE819.7070006@openoffice.org>
 <0c5601ccb53b$02e96590$08bc30b0$@16degrees.com.au>
 <4EE077F6.8010100@googlemail.com>
 <04ed01ccb588$c48e1f90$4daa5eb0$@16degrees.com.au>
In-Reply-To: <04ed01ccb588$c48e1f90$4daa5eb0$@16degrees.com.au>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 12/8/11 10:07 AM, Gavin McDonald wrote:
>
>
>> -----Original Message-----
>> From: Jürgen Schmidt [mailto:jogischmidt@googlemail.com]
>> Sent: Thursday, 8 December 2011 6:40 PM
>> To: ooo-dev@incubator.apache.org
>> Subject: Re: Extensions and templates
>>
>> On 12/8/11 12:50 AM, Gavin McDonald wrote:
>>>
>>>
>>>> -----Original Message-----
>>>> From: Andrea Pescetti [mailto:pescetti@openoffice.org]
>>>> Sent: Thursday, 8 December 2011 8:27 AM
>>>> To: ooo-dev@incubator.apache.org
>>>> Subject: Re: Extensions and templates
>>>>
>>>> On 29/11/2011 Rob Weir wrote:
>>>>> ==Option 1: Remain at OSUOSL==
>>>>> We could remain with OSUOSL hosting.  However, the existing site is
>>>>> very unstable.
>>>>
>>>> This would be best both for short and long term. In the short term,
>>>> it provides continuity of service and it doesn't break existing
>>>> links. In the long term, the Drupal instance can be updated and
>>>> extended (it's not easy, but it is something that would have fairly
>>>> high chances of
>>>> success) to get it to be something like the new model (distributed
>>>> repositories) you describe in step 5.
>>>>
>>>
>>> Sorry, OSUOSL don’t want anything to do with these any longer, I
>>> thought folks got the hint when they turned off monitoring and no
>>> longer look at the issues of the host, but rather restart it only when
>>> prompted. The hosts themselves cannot cope with all the memory and cpu
>> these are consuming all the time, let alone the bandwidth.
>>>
>>> This is no longer an option.
>>>
>>>> An important point that everybody seems to miss is that the
>>>> instability of the current Extensions site
>>>> http://extensions.services.openoffice.org/ is, very likely, unrelated
>>>> to Drupal. The underlying Drupal instance is rather sound (and it
>>>> perfectly managed to sustain the traffic in 2010, which should not be
>>>> significantly different from 2011); from the behavior of the site, it
>>>> definitely seems to me that the instability is due to other components,
>> like the caching server (Varnish) in front of it or other caching mechanisms.
>>>
>>> I very much disagree. Caching can be tweaked for sure, but I've had a
>>> look around the drupal sites and it is not optimal to say the least.
>>>
>>>>
>>>> A second very important point is that we need to get the Extensions
>>>> and Templates source code (two different codebases) under the SGA;
>>>> while Drupal itself is GPL, Thorsten Bosbach while working at Oracle
>>>> created a lot of custom PHP code for the two sites. This code, as far
>>>> as I know, has never been released. Access to the source code is a
>>>> prerequisite for any possible analysis/improvement of the website.
>>>>
>>>>> ==Option 2: Move Critical extensions to stable host==
>>>>
>>>> Indeed, as you write, this would be an extreme option.
>>>
>>> More extreme would be to do nothing, as you'll end up with nothing.
>>>
>>>>
>>>>> ==Option 3: Clone OSUOSL repositories to another host==
>>>>
>>>> This is not significantly different from Option 1; i.e., if there are
>>>> other hosting options available the mere cloning of the site would
>>>> not take long, but again the problem is not with the site but with caching.
>>>
>>> Do not blame caches for poor performance. the caches are improving a
>>> bad situation, they can be tweaked to improve further.
>>>
>>>>
>>>> Note that, since the Templates site has already been ported to Drupal
>>>> 6 using the so called "code-driven development" technique, that
>>>> source code would allow to install an empty pre-configured clone of
>>>> the Templates site anywhere. This would be extremely useful for testing.
>>>>
>>>>> ==Option 4: Host repositories elsewhere, using new UI==
>>>>
>>>> As I used to say, everybody who thinks that the Extensions or
>>>> Templates sites can be replaced easily has never tried submitting a
>> template!
>>>> Thorsten did a lot of customization work on the two sites; any
>>>> replacement would provide a largely inferior user experience.
>>>
>>> I think you don’t think very highly of other peoples abilities, a poor outlook.
>>>
>>>>
>>>>> ==Option 5: Re-architect the Repositories== This is the option I
>>>>> personally favor for long term. ...
>>>>> This would allow multiple
>>>>> repositories to look and behave identically from the data perspective.
>>>>
>>>> This is an interesting long term solution indeed, but I see it
>>>> feasible as a
>>>> (complex) version of Option 1-3; i.e., we obtain the current codebase
>>>> with the aim of updating it and extending it in this direction.
>>>>
>>>>> The other thing this approach does is separate the extension
>>>>> metadata from the actual licensed extension.  If we wanted to have a
>>>>> canonical repository of registered extensions, but without actually
>>>>> hosting or storing the extensions, then that should be OK.  We're
>>>>> hosting URL's to resources.  We're not distributing code.
>>>>
>>>> This would offer some advantages, but I see advantages in offering
>>>> hosting for extensions too. The current Extensions site offers both
>>>> options (host there or externally), but if I recall correctly some
>>>> automatic mechanisms, like autogeneration of the update URL, only work
>> if the local hosting is used.
>>>>
>>>
>>> Having spoken to OSUOSL, having looked around the machines and
>>> services in question and having looked at and been told of the
>>> excessive bandwidth (and that is MUST stop), here is the route I intend to
>> take:
>>>
>>> 1. Move the services to a newer more modern host at the ASF
>>> (temporary) 2. BandAid the installation to stabilise it for the short
>>> term (this is still more work than it sounds) 3. Stick Apache TrafficServer in
>> front (not varnish) to improve response times / caching.
>>> 4. Go with the choice of Option 5. that is, to allow the hosting and
>> downloading of the templates
>>>      and extensions to be with the 3rd party authors. We will hold master
>> copies, and provide metadata
>>>     and links to the download locations / master sites, but we will not allow
>> downloading directly.
>>>     This will solve the excessive bandwidth issues longer term. I intend to
>> start the work of this sometime
>>>     in January.
>>
>> we should keep in mind that not every extension or template developer has
>> the opportunity to host the extension or template themselves.
>>
>> And then we have potentially the problem that the third party sides are not
>> available when users try to download. Very poor user experience. Ok the
>> moment it is also bad but we are looking for a long term solution.
>>
>> If Apache is not able to host such a repository, we can of course think of
>> multiple repositories in the future.
>>
>> The Apache one would be the default. And here we can host extension that
>> are Apache conform and potentially hosted on our svn or apache-extras.
>>
>>>
>>> If you or anyone else here has any complaints or issues or further
>>> idea, please bring them to the infra team now as I intend to get cracking in
>> this very soon, the status quo can not continue, for benefit of all.
>>>
>>> Help welcomed at any step of the way.
>>>
>>> (Note that moving services from OSUOSL hosts to the ASF hosts does
>>> nothing to solved the bandwidth issues because the ASF servers are
>>> also OSUOSL hosted!)
>>
>> ups, that is not really promising when i think of
>> - a future download of OOo binaries.
>> - the svn performance
>> - the wiki performance (confluence wiki)
>
> That was not the most encouraging comment you could provide this thread
> considering the work I've just volunteered to do to resolve this mess.

i don't know why you feel offended, at least it seems so. It's nothing 
personal and I very much appreciate your work. And I hope others 
appreciate my work (mainly on the code at the moment) as well.

>
> The overall bandwidth at OSUOSl is not in jeopardy but they are also not
> infinite, one must use them wisely.
>
> Please refrain from being negative.
i don't want to be too negative it's simply my personal impression and 
observation (at least the performance).

Juergen