Mailing-List: contact ooo-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: ooo-dev@incubator.apache.org
Received-SPF: pass (athena.apache.org: domain of dennis.hamilton@acm.org
 designates 75.98.160.130 as permitted sender)
Reply-To: <dennis.hamilton@acm.org>
From: "Dennis E. Hamilton" <dennis.hamilton@acm.org>
To: <ooo-dev@incubator.apache.org>
References: <02ff01cc6d09$b352c0e0$19f842a0$@acm.org>
	<4E67F7FA.5010807@gmail.com>
 <CAMK9kTiOku-D7ciheZkaW5X1qUpjO0jQHE=jhe-69-7810ojNQ@mail.gmail.com>
In-Reply-To: 
 <CAMK9kTiOku-D7ciheZkaW5X1qUpjO0jQHE=jhe-69-7810ojNQ@mail.gmail.com>
Subject: RE: [wiki] Migration - A TerryE Clipping Collection [LONG]
Date: Wed, 7 Sep 2011 19:43:42 -0700
Organization: NuovoDoc
Message-ID: <022c01cc6dd1$1fedeaa0$5fc9bfe0$@acm.org>
MIME-Version: 1.0
Content-Type: text/plain;
	charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Thread-Index: AQKTX5lG/Wuwm3oHQCodSMqmhyb/rgJPW4zYAaeEu/2TlSOWsA==
Content-Language: en-us

Alexandro,

I would follow Terry's advice and bring over the MediaWiki as close to =
as-is as we can make it, when we are ready to substitute for the one =
running beneath OpenOffice.org.  What works for me is that the attention =
to risk management and especially removing front-loading from the =
project appeals to me.  It is the agile principle of first doing the =
least that could possibly work.

Terry indicated that installing MediaWiki could be a stopgap.  Having =
brought over the platform and the content secures ease for review and =
migration to Confluence in a careful way without concern for the lights =
going out underneath OpenOffice.org.

Anything that can help to simply rehost OpenOffice.org and have it on =
operating sites that the Apache OOo Podling can maintain and transform =
seems best.  That is when all of us can breathe easier.  And the users =
of OpenOffice.org will be served and disrupted as little as possible.  =
Then we incubate the site and its services along with all other =
incubation.

The key difficulty in bringing up the wiki completely is the need to =
upgrade to MW 1.17.  I'm hoping that there is no rush.  A volunteer able =
to work through that is needed.=20

 - Dennis

PS: I clipped out relevant Terry Notes below:

-----Original Message-----
From: acolorado@gmail.com [mailto:acolorado@gmail.com] On Behalf Of =
Alexandro Colorado
Sent: Wednesday, September 07, 2011 17:12
To: ooo-dev@incubator.apache.org
Subject: Re: [wiki] Migration - A TerryE Clipping Collection [LONG]

On Wed, Sep 7, 2011 at 6:02 PM, Kay Schenk <kay.schenk@gmail.com> wrote:

> Dennis--
>
> Thanks very much for sending this. I discovered that Terry had =
actually
> done a great job of documenting what had already occurred and some =
thoughts
> on what needed to be done on this wiki page...
>
>
>
> =
https://cwiki.apache.org/confluence/display/OOOUSERS/Community+Wiki+Servi=
ces
>
> So, I took the liberty of documenting where we are now -- also =
exploring
> Confluence.
>

I also have not kept up with the discussion but reading Terry's comments
there seem to be a very huge task to migrate out of Mediawiki. However =
then
I see that "due to lack of Mediawiki support - we are moving to =
Confluence".
Isnt this exactly what Terry describe why would this be difficult to do?
I also don't see any support for confluence products to match what =
mediawiki
had. At least any solutions to keep the functionality for their content.

Can you please explain how will you migrate addressing Terry's issues on
migration?


>> *** 2011-07-31-13:53 All times PDT (UTC-0700), TerryE - Wiki Size
>>
>> The OOo wiki contains 10,521 content pages and 11.338 uploaded files.
>> These form a critical service to the end-user community.  Note that
>> the cwiki markup format doesn't support many of the MediaWiki
>> specific markups in this content.
>>
>> *** 2011-07-31-14:36 TerryE - Concerns for Conversion to CWiki
>>
>> [T]he challenge here is that the cwiki markup grammer is different to
>> that of mediawiki and largely a functional subset.  Worse, mediawiki
>> supports extensions to allow you to extend this. The OOo
>> implementation currently uses *42* such extensions.  One of these was
>> custom-developed by the German OOo team.  With over 10,000 pages of
>> content, sentencing this lot and even if we establish a migration
>> ruleset to map 90% of the non-conformities, we still looking at a
>> MAJOR project.
>>
[ ... ]

>> *** 2011-08-01-09:53 TerryE - More on Confluence Migration
>>
>> *I* have looked at [migration to Confluence] and it *will* be hard. I
>> didn't say impossible. ... Just look at
>>
>> *  http://wiki.services.openoffice.org/wiki/Special:Statistics *
>> http://wiki.services.openoffice.org/wiki/Special:Version
>>
>> for the volumetrics and MediaWiki extensions used.  If you google
>> "mediawiki confluence migration" then you will see that isn't a
>> trivial exercise even for wiki using standard MW with no extensions:
>> it would involve person-years of effort.  I have suggested ...
>> rehosting but on a sustain basis for the wiki.  I can set all of this
>> up, if agreeable to the project.  It doesn't need further material
>> resources from the Apache team.  This would at least buy us the time
>> to do a proper migration plan and resource it.
>>
[ ... ]

>> *** 2011-08-03-08:07 Terry Ellison - Reaching the Wiki Users, Legal
>> Concerns
>>
>>> [...] Is there a announcement list or some other mechanism to send
>>> an email to every registered wiki user?
>>>
>>
>> At a technical level, it's simple to run a query dumping all of the
>> mail addresses of contributors to the wiki.  I've just done a few on
>> my local VM which has a snapshot of  the prod wiki as at Thursday/Fri
>> night IIRC.
>>
>> * There are 34,969 registered users.  Of which * 3,675 have made
>> contributions.  There is no need to contact those who haven't * 3,623
>> have registered email addresses and have made 182,677 contributions *
>> 52 have no registered email addresses and have made 153 contributions
>> (prob dating back to the early days when email registration and
>> confirmation wasn't mandatory
>>
[ ... ]

>> *** 2011-08-04-01:36 Terry Ellison - Moving the WIki "As-Is" for Now
>>
>> I am working with the rest of the project to migrate the forums and
>> the wiki "as-is" for now.
>>
>> .  As far as the wiki goes in the medium to longer term, the project
>> may decide to move some material to cwiki, but this is work in
>> progress.
 [ ... ]

>> *** 2011-08-08-02:22 Terry Ellison - Derisking Migration/Staging -
>> Separating Infrastructure/Platform and Content
>>
>> Thank-you all for your replies so far.  I will work through and
>> respond to individual points in thread at the appropriate branch, but
>> here I just wanted to propose that we divide the Wiki migration into
>> two separate but task areas:
>>
>> * Migration of the infrastructure service.  What drives this is if
>> Oracle decides, say, that the current hardware is being turned off on
>> the XXth of Aug, then we need to move the service to a box in
>> Apache.org.  The issues here are largely technical heavily involve
>> the infrastructure team.  We need to be able to move quickly on this
>> and few of the DL contributors have strong opinions here. They just
>> want the 'magic to happen'.  Yes, there are a few decisions to be
>> made, but the questions are a little esoteric for most DL members.  I
>> offer to continue to lead this work area.
>>
>> * Migration of the wiki content / policy.  Here we have a range of
>> widely and strongly held views and the DL seems to be acting as a
>> debating shop without convergence to consensus on some points. This
>> debate could still be continuing in 3 months time as far as I can
>> see.  I don't want it to get in the way of infrastructure migration
>> -- except when /absolutely/ necessary.  It also makes sense to break
>> this task area further if this means that we can make progress in
>> some areas, even if stalled in others.
>>
>> but from an infrastructure viewpoint I don't really care this
>> branding process can be encapsulated and detached.
>>
>> * We make the pre-prod wiki available to the "branding team" * The
>> branding team agree and implement the branding changes that the
>> project wants to have in place for cut-over day.  In practice this
>> could involve the creation or modification of content in the Main,
>> File, MediaWiki, OpenOffice.org Wiki and Template Namespaces * I can
>> use standard MW audit functionality to capture a list of all such
>> pages, and then standard MW export functionality to create an XML.GZ
>> dump of this work. * We then import the "live" wiki as part of
>> cut-over. * I then reimport the above XML.GZ to reapply the branding
>> changes to the migrated wiki.
>>
>> ... [M]y suggestion is to keep the infrastructure and content issues
>> separate.  I can lead on the first.  We need someone who can lead on
>> the second. His or her strengths should be that he/she has experience
>> of delivery -- that is can make things happen -- and has a good
>> working knowledge of more advance MW features such as templates, etc.
>> I would think that either Drew or TJ would be well qualified, but
>> that is only my personal suggestion.
[ ... ]

>> *** 2011-08-08-03:33 Terry Ellison - Risks of Changing DBMS along
>> with Migration
>>
>> There are two factors against an early move to PostgreSQL:  (i) The
>> MediaWiki cavaets on its use (here
>> <http://www.mediawiki.org/wiki/Manual:PostgreSQL>  and as all
>> experienced Wikipedian's do look at the associated talk page here
>> <http://www.mediawiki.org/wiki/Manual_talk:PostgreSQL>; this has a
>> couple of interesting references which cookbook the conversion).
>> (ii) The extra work involved. This is not only the D/B migration, but
>> also a MW version upgrade 15.1 ->  17.
>>
>> To be honest I am uncomfortable doing this as part of this immediate
>> "continuity of service" migration.  My suggestion is that if the
>> wiki looks as if it is going to have a long term place within  the
>> project then we should revisit this as part of an in-service
>> improvement program in, say, 6-12 months.
[ ... ]

>> *** 2011-08-10 15:12 Terry Ellison TEST/PRE-PRODUCTION WIKI INSTANCE
>> PROGRESS
>>
>> Just a quick update on progress on the wiki instance.  This is now up
>> in preprod mode  on the subdomain ooo-wiki.  This is based on a
>> snapshot taken at 4am yesterday morning.  The server needs further
>> tuning, but functionally its there.  We do have some outstanding code
>> issues (due to a change in the functionality of the PHP
>> routine*call_user_func* between Version 5.3 and 5.2 which is not
>> backwards compatible.  I'll sort this out tomorrow).
>>
>> Also note as per my previous email, all existing account have had
>> there passwords mangled, so access is guest-mode only at the moment.
>> Feel free to have a look, but updates and no service commitments
>> yet.
[ ... ]

>> *** 2011-08-11-01:38 Terry Ellison, BREAKAGE IN SEPARATING WIKI FROM
>> OO.O
>>
>>> I found one bug: On the wiki left side lowest frame where store
>>> links for other languages main pages, the links points to
>>> http://wiki.services.openoffice.org, not to the
>>> http://ooo-wiki.apache.org/.
>>>
>>
>> You are correct, and in fact there are a lot of references to
>> *.OpenOffice.org in the content.  However, there isn't a production
>> instance and there little point in changing these until (1) we agree
>> what such "dirty" references are going to be changed to, and (2) we
>> cut over for real.  Remember that any changes that we made on this
>> instance will be lost when we bring over the live [database].
[ ... ]

>> *** 2011-08-25-08:59 Terry Ellison, REQUEST FOR BASELINE AGREEMENTS
>> TO MOVE AHEAD [abridged 2011-09-08T02:33Z by dh]
>>
>> *    The prod wiki is v1.15.1 that at an N-3 major release level
>> (that's 30 months old: two major and 10 minor revisions behind the
>> current supported).  This also runs on PHP 5.2.0.
>> [...]
>> The PHP 5.3 introduced extra checking to remove an area of tolerance
>> the PHP 5.x<3 allowed.  This was to do with when and how parameters
>> can be passed by reference under curtain circumstances.  So moving a
>> code base from 5.2 to 5.3 involved a lot of work identifying and
>> eliminating this mis-codings.  This was done by the MediaWiki team in
>> MW v1.16.  I had planned to move to MW v1.15.5 (the last stable
>> 1.15.x) as our baseline and I've done this work integrating it with
>> Apache Traffic Server (ATS) and our LAMP stack.  This is stable and
>> performant enough to show that we are good.  However, I have only
>> identified and bug-fixed the main path 5.2->5.3 coding issues.
>> During my testing I have subsequently discovered others and there are
>> undoubtedly more to find.  I've also discussed this with the MW devs
>> on the MW IRC channel.  Given this, the consensus in the @infra team
>> (me included) is that we should bite the bullet now and move to
>> current MW 1.17.0 even given the extra work. There are some
>> performance risks associated with MW 1.17.0 which we need to
>> mitigate.  However, given that we've already got a complete
>> LAMP+ATS+MV in an ESX hosted VM performing like a dingbat, we really
>> only face the 1.15.5 ->  1.17.0 issues in this step.
>>
>> * *DECISION*: Upgrade the ooo-wiki MediaWiki(MV) + all extensions to
>> MW v1.17.0.
>>
>> * *DECISION*: I have agreed with infrastructure that we will keep 1
>> core on "standby" so we can up the VM to a 2-core VM if we are
>> seeing unacceptable performance problems with one-core.
>>
>> * *DECISION*: We will cut over the wiki and the forums with the
>> content as-is and implement branding and access control changes
>> within the a.o infrastructure when volunteers come on-stream to
>> resource this.  This is the standard "transfer then clean-up"
>> approach adopted when a migration is time critical.
>> [ ... ]
>> *     We have some further caching tweaks on the interaction of the
>> MediaWiki [application] with the ATS HTTP reverse proxy cache, but
>> these are probably nice-to-have than essential.  More to the point
>> these need to be done on a system will production load patterns.
>>
>> * *DECISION*. We will defer such tuning until post go-live.
>>
>> *** END OF THE WIKI NOTES ***
>>
>>
> --
> =
------------------------------------------------------------------------
> MzK
>
> "There's something about the sound of a train
>  that's very romantic and nostalgic and hopeful."
>                               -- Paul Simon
>


--=20
*Alexandro Colorado*
*OpenOffice.org* Espa=C3=B1ol
http://es.openoffice.org
fingerprint: E62B CF77 1BEA 0749 C0B8 50B9 3DE6 A84A 68D0 72E6