Mailing-List: contact ooo-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: ooo-dev@incubator.apache.org
Received-SPF: pass (athena.apache.org: local policy)
Reply-To: <gavin@16degrees.com.au>
From: "Gavin McDonald" <gavin@16degrees.com.au>
To: <ooo-dev@incubator.apache.org>
References: 
 <CAP-ksohR7w-=1v9-D=yWjNuebURp94Txm1iMjinUamXCPY0Qjw@mail.gmail.com>
 <4EDFE819.7070006@openoffice.org>
 <0c5601ccb53b$02e96590$08bc30b0$@16degrees.com.au>
 <4EE077F6.8010100@googlemail.com>
In-Reply-To: <4EE077F6.8010100@googlemail.com>
Subject: RE: Extensions and templates
Date: Thu, 8 Dec 2011 19:07:10 +1000
Organization: 16 degrees
Message-ID: <04ed01ccb588$c48e1f90$4daa5eb0$@16degrees.com.au>
MIME-Version: 1.0
Content-Type: text/plain;
	charset="utf-8"
Content-Transfer-Encoding: quoted-printable
thread-index: AQLWsXEVkQSALmmf3mJ4ohdx0OQy9gIhjsaTAfVDXHQCbbEu5pOJif0w
Content-Language: en-au


> -----Original Message-----
> From: J=C3=BCrgen Schmidt [mailto:jogischmidt@googlemail.com]
> Sent: Thursday, 8 December 2011 6:40 PM
> To: ooo-dev@incubator.apache.org
> Subject: Re: Extensions and templates
>=20
> On 12/8/11 12:50 AM, Gavin McDonald wrote:
> >
> >
> >> -----Original Message-----
> >> From: Andrea Pescetti [mailto:pescetti@openoffice.org]
> >> Sent: Thursday, 8 December 2011 8:27 AM
> >> To: ooo-dev@incubator.apache.org
> >> Subject: Re: Extensions and templates
> >>
> >> On 29/11/2011 Rob Weir wrote:
> >>> =3D=3DOption 1: Remain at OSUOSL=3D=3D
> >>> We could remain with OSUOSL hosting.  However, the existing site =
is
> >>> very unstable.
> >>
> >> This would be best both for short and long term. In the short term,
> >> it provides continuity of service and it doesn't break existing
> >> links. In the long term, the Drupal instance can be updated and
> >> extended (it's not easy, but it is something that would have fairly
> >> high chances of
> >> success) to get it to be something like the new model (distributed
> >> repositories) you describe in step 5.
> >>
> >
> > Sorry, OSUOSL don=E2=80=99t want anything to do with these any =
longer, I
> > thought folks got the hint when they turned off monitoring and no
> > longer look at the issues of the host, but rather restart it only =
when
> > prompted. The hosts themselves cannot cope with all the memory and =
cpu
> these are consuming all the time, let alone the bandwidth.
> >
> > This is no longer an option.
> >
> >> An important point that everybody seems to miss is that the
> >> instability of the current Extensions site
> >> http://extensions.services.openoffice.org/ is, very likely, =
unrelated
> >> to Drupal. The underlying Drupal instance is rather sound (and it
> >> perfectly managed to sustain the traffic in 2010, which should not =
be
> >> significantly different from 2011); from the behavior of the site, =
it
> >> definitely seems to me that the instability is due to other =
components,
> like the caching server (Varnish) in front of it or other caching =
mechanisms.
> >
> > I very much disagree. Caching can be tweaked for sure, but I've had =
a
> > look around the drupal sites and it is not optimal to say the least.
> >
> >>
> >> A second very important point is that we need to get the Extensions
> >> and Templates source code (two different codebases) under the SGA;
> >> while Drupal itself is GPL, Thorsten Bosbach while working at =
Oracle
> >> created a lot of custom PHP code for the two sites. This code, as =
far
> >> as I know, has never been released. Access to the source code is a
> >> prerequisite for any possible analysis/improvement of the website.
> >>
> >>> =3D=3DOption 2: Move Critical extensions to stable host=3D=3D
> >>
> >> Indeed, as you write, this would be an extreme option.
> >
> > More extreme would be to do nothing, as you'll end up with nothing.
> >
> >>
> >>> =3D=3DOption 3: Clone OSUOSL repositories to another host=3D=3D
> >>
> >> This is not significantly different from Option 1; i.e., if there =
are
> >> other hosting options available the mere cloning of the site would
> >> not take long, but again the problem is not with the site but with =
caching.
> >
> > Do not blame caches for poor performance. the caches are improving a
> > bad situation, they can be tweaked to improve further.
> >
> >>
> >> Note that, since the Templates site has already been ported to =
Drupal
> >> 6 using the so called "code-driven development" technique, that
> >> source code would allow to install an empty pre-configured clone of
> >> the Templates site anywhere. This would be extremely useful for =
testing.
> >>
> >>> =3D=3DOption 4: Host repositories elsewhere, using new UI=3D=3D
> >>
> >> As I used to say, everybody who thinks that the Extensions or
> >> Templates sites can be replaced easily has never tried submitting a
> template!
> >> Thorsten did a lot of customization work on the two sites; any
> >> replacement would provide a largely inferior user experience.
> >
> > I think you don=E2=80=99t think very highly of other peoples =
abilities, a poor outlook.
> >
> >>
> >>> =3D=3DOption 5: Re-architect the Repositories=3D=3D This is the =
option I
> >>> personally favor for long term. ...
> >>> This would allow multiple
> >>> repositories to look and behave identically from the data =
perspective.
> >>
> >> This is an interesting long term solution indeed, but I see it
> >> feasible as a
> >> (complex) version of Option 1-3; i.e., we obtain the current =
codebase
> >> with the aim of updating it and extending it in this direction.
> >>
> >>> The other thing this approach does is separate the extension
> >>> metadata from the actual licensed extension.  If we wanted to have =
a
> >>> canonical repository of registered extensions, but without =
actually
> >>> hosting or storing the extensions, then that should be OK.  We're
> >>> hosting URL's to resources.  We're not distributing code.
> >>
> >> This would offer some advantages, but I see advantages in offering
> >> hosting for extensions too. The current Extensions site offers both
> >> options (host there or externally), but if I recall correctly some
> >> automatic mechanisms, like autogeneration of the update URL, only =
work
> if the local hosting is used.
> >>
> >
> > Having spoken to OSUOSL, having looked around the machines and
> > services in question and having looked at and been told of the
> > excessive bandwidth (and that is MUST stop), here is the route I =
intend to
> take:
> >
> > 1. Move the services to a newer more modern host at the ASF
> > (temporary) 2. BandAid the installation to stabilise it for the =
short
> > term (this is still more work than it sounds) 3. Stick Apache =
TrafficServer in
> front (not varnish) to improve response times / caching.
> > 4. Go with the choice of Option 5. that is, to allow the hosting and
> downloading of the templates
> >     and extensions to be with the 3rd party authors. We will hold =
master
> copies, and provide metadata
> >    and links to the download locations / master sites, but we will =
not allow
> downloading directly.
> >    This will solve the excessive bandwidth issues longer term. I =
intend to
> start the work of this sometime
> >    in January.
>=20
> we should keep in mind that not every extension or template developer =
has
> the opportunity to host the extension or template themselves.
>=20
> And then we have potentially the problem that the third party sides =
are not
> available when users try to download. Very poor user experience. Ok =
the
> moment it is also bad but we are looking for a long term solution.
>=20
> If Apache is not able to host such a repository, we can of course =
think of
> multiple repositories in the future.
>=20
> The Apache one would be the default. And here we can host extension =
that
> are Apache conform and potentially hosted on our svn or apache-extras.
>=20
> >
> > If you or anyone else here has any complaints or issues or further
> > idea, please bring them to the infra team now as I intend to get =
cracking in
> this very soon, the status quo can not continue, for benefit of all.
> >
> > Help welcomed at any step of the way.
> >
> > (Note that moving services from OSUOSL hosts to the ASF hosts does
> > nothing to solved the bandwidth issues because the ASF servers are
> > also OSUOSL hosted!)
>=20
> ups, that is not really promising when i think of
> - a future download of OOo binaries.
> - the svn performance
> - the wiki performance (confluence wiki)

That was not the most encouraging comment you could provide this thread
considering the work I've just volunteered to do to resolve this mess.

The overall bandwidth at OSUOSl is not in jeopardy but they are also not
infinite, one must use them wisely.

Please refrain from being negative.

Gav...

>=20
> Juergen
>=20
> >
> > Gav...
> >
> >
> >> Regards,
> >>     Andrea.
> >