Mailing-List: contact ooo-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: ooo-dev@incubator.apache.org
Received-SPF: pass (nike.apache.org: domain of jogischmidt@googlemail.com
 designates 74.125.82.43 as permitted sender)
Message-ID: <4EE077F6.8010100@googlemail.com>
Date: Thu, 08 Dec 2011 09:40:22 +0100
From: =?UTF-8?B?SsO8cmdlbiBTY2htaWR0?= <jogischmidt@googlemail.com>
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7;
 rv:8.0) Gecko/20111105 Thunderbird/8.0
MIME-Version: 1.0
To: ooo-dev@incubator.apache.org
Subject: Re: Extensions and templates
References: 
 <CAP-ksohR7w-=1v9-D=yWjNuebURp94Txm1iMjinUamXCPY0Qjw@mail.gmail.com>
 <4EDFE819.7070006@openoffice.org>
 <0c5601ccb53b$02e96590$08bc30b0$@16degrees.com.au>
In-Reply-To: <0c5601ccb53b$02e96590$08bc30b0$@16degrees.com.au>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 12/8/11 12:50 AM, Gavin McDonald wrote:
>
>
>> -----Original Message-----
>> From: Andrea Pescetti [mailto:pescetti@openoffice.org]
>> Sent: Thursday, 8 December 2011 8:27 AM
>> To: ooo-dev@incubator.apache.org
>> Subject: Re: Extensions and templates
>>
>> On 29/11/2011 Rob Weir wrote:
>>> ==Option 1: Remain at OSUOSL==
>>> We could remain with OSUOSL hosting.  However, the existing site is
>>> very unstable.
>>
>> This would be best both for short and long term. In the short term, it
>> provides continuity of service and it doesn't break existing links. In the long
>> term, the Drupal instance can be updated and extended (it's not easy, but it
>> is something that would have fairly high chances of
>> success) to get it to be something like the new model (distributed
>> repositories) you describe in step 5.
>>
>
> Sorry, OSUOSL don’t want anything to do with these any longer, I thought folks got the
> hint when they turned off monitoring and no longer look at the issues of the host, but
> rather restart it only when prompted. The hosts themselves cannot cope with all the
> memory and cpu these are consuming all the time, let alone the bandwidth.
>
> This is no longer an option.
>
>> An important point that everybody seems to miss is that the instability of the
>> current Extensions site http://extensions.services.openoffice.org/ is, very
>> likely, unrelated to Drupal. The underlying Drupal instance is rather sound
>> (and it perfectly managed to sustain the traffic in 2010, which should not be
>> significantly different from 2011); from the behavior of the site, it definitely
>> seems to me that the instability is due to other components, like the caching
>> server (Varnish) in front of it or other caching mechanisms.
>
> I very much disagree. Caching can be tweaked for sure, but I've had a look around
> the drupal sites and it is not optimal to say the least.
>
>>
>> A second very important point is that we need to get the Extensions and
>> Templates source code (two different codebases) under the SGA; while
>> Drupal itself is GPL, Thorsten Bosbach while working at Oracle created a lot of
>> custom PHP code for the two sites. This code, as far as I know, has never
>> been released. Access to the source code is a prerequisite for any possible
>> analysis/improvement of the website.
>>
>>> ==Option 2: Move Critical extensions to stable host==
>>
>> Indeed, as you write, this would be an extreme option.
>
> More extreme would be to do nothing, as you'll end up with nothing.
>
>>
>>> ==Option 3: Clone OSUOSL repositories to another host==
>>
>> This is not significantly different from Option 1; i.e., if there are other hosting
>> options available the mere cloning of the site would not take long, but again
>> the problem is not with the site but with caching.
>
> Do not blame caches for poor performance. the caches are improving a bad situation,
> they can be tweaked to improve further.
>
>>
>> Note that, since the Templates site has already been ported to Drupal 6 using
>> the so called "code-driven development" technique, that source code would
>> allow to install an empty pre-configured clone of the Templates site
>> anywhere. This would be extremely useful for testing.
>>
>>> ==Option 4: Host repositories elsewhere, using new UI==
>>
>> As I used to say, everybody who thinks that the Extensions or Templates
>> sites can be replaced easily has never tried submitting a template!
>> Thorsten did a lot of customization work on the two sites; any replacement
>> would provide a largely inferior user experience.
>
> I think you don’t think very highly of other peoples abilities, a poor outlook.
>
>>
>>> ==Option 5: Re-architect the Repositories== This is the option I
>>> personally favor for long term. ...
>>> This would allow multiple
>>> repositories to look and behave identically from the data perspective.
>>
>> This is an interesting long term solution indeed, but I see it feasible as a
>> (complex) version of Option 1-3; i.e., we obtain the current codebase with
>> the aim of updating it and extending it in this direction.
>>
>>> The other thing this approach does is separate the extension metadata
>>> from the actual licensed extension.  If we wanted to have a canonical
>>> repository of registered extensions, but without actually hosting or
>>> storing the extensions, then that should be OK.  We're hosting URL's
>>> to resources.  We're not distributing code.
>>
>> This would offer some advantages, but I see advantages in offering hosting
>> for extensions too. The current Extensions site offers both options (host
>> there or externally), but if I recall correctly some automatic mechanisms, like
>> autogeneration of the update URL, only work if the local hosting is used.
>>
>
> Having spoken to OSUOSL, having looked around the machines and services in question
> and having looked at and been told of the excessive bandwidth (and that is MUST stop),
> here is the route I intend to take:
>
> 1. Move the services to a newer more modern host at the ASF (temporary)
> 2. BandAid the installation to stabilise it for the short term (this is still more work than it sounds)
> 3. Stick Apache TrafficServer in front (not varnish) to improve response times / caching.
> 4. Go with the choice of Option 5. that is, to allow the hosting and downloading of the templates
>     and extensions to be with the 3rd party authors. We will hold master copies, and provide metadata
>    and links to the download locations / master sites, but we will not allow downloading directly.
>    This will solve the excessive bandwidth issues longer term. I intend to start the work of this sometime
>    in January.

we should keep in mind that not every extension or template developer 
has the opportunity to host the extension or template themselves.

And then we have potentially the problem that the third party sides are 
not available when users try to download. Very poor user experience. Ok 
the moment it is also bad but we are looking for a long term solution.

If Apache is not able to host such a repository, we can of course think 
of multiple repositories in the future.

The Apache one would be the default. And here we can host extension that 
are Apache conform and potentially hosted on our svn or apache-extras.

>
> If you or anyone else here has any complaints or issues or further idea, please bring them to the infra team now as I
> intend to get cracking in this very soon, the status quo can not continue, for benefit of all.
>
> Help welcomed at any step of the way.
>
> (Note that moving services from OSUOSL hosts to the ASF hosts does nothing to solved the bandwidth
> issues because the ASF servers are also OSUOSL hosted!)

ups, that is not really promising when i think of
- a future download of OOo binaries.
- the svn performance
- the wiki performance (confluence wiki)

Juergen

>
> Gav...
>
>
>> Regards,
>>     Andrea.
>