incubator-ooo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dennis E. Hamilton" <dennis.hamil...@acm.org>
Subject [wiki] Migration - A TerryE Clipping Collection [LONG]
Date Wed, 07 Sep 2011 02:56:10 GMT
I compiled a LONG chronological clipping set of notes from Terry on from conception to when
he stopped posting on the issues and thinking around Wiki migration.  

I'm breaking a thread here. There is no need to comment on any of this, it is for reference
and mining for things to not overlook, collate onto the wiki, etc. 

The selections are mine, the list is long, and it probably works better distilled onto a wiki.
 This is the collection phase.

I am staying as close to the architectural and technical considerations that I can, and not
follow into areas where there were policy spats of one kind or another.  I thought I might
have to look at others posts too, but generally, I could find enough in what Terry was responding
to to provide continuity.

It is interesting that we are now re-hashing some of the things that Terry covered at least
one starting over a month ago.

NOTE: Terry also provided OOOUSER Material

*** 2011-07-31-13:53 All times PDT (UTC-0700), TerryE - Wiki Size

The OOo wiki contains 10,521 content pages and  
11.338 uploaded files.  These form a critical service to the end-user 
community.  Note that the cwiki markup format doesn't support many of 
the MediaWiki specific markups in this content.

*** 2011-07-31-14:36 TerryE - Concerns for Conversion to CWiki

[T]he challenge here is that the cwiki markup 
grammer is different to that of mediawiki and largely a functional 
subset.  Worse, mediawiki supports extensions to allow you to extend 
this. The OOo implementation currently uses *42* such extensions.  One 
of these was custom-developed by the German OOo team.  With over 10,000 
pages of content, sentencing this lot and even if we establish a 
migration ruleset to map 90% of the non-conformities, we still looking 
at a MAJOR project.

*** 2011-07-31-15:07 Terry Ellison - Formulating the Hardware/Software Stack

Both the OOo forums and the OOo wiki can run on a standard LAMP stack.  
[ ... ]

The wiki is a bit more of a hog and need 4-cores, but it currently 
doesn't use a PHP opcode cache and this would halve this load.  Most of 
the access is guest, so using a squid or varnish front-end will drop 
this significantly.

The simplest way to provide this service would be to use VMs and AFAIK 
this is a model that the a.o infrastructure guys understand.

*** 2011-07-31-16:02 TerryE - Migrating the Wiki, Concerns not to Overlook

As far as the wiki goes, this template has been more heavily customised 
again using standard MediaWiki hooks and facilities.  Switching logos 
and banners would be reasonably straight forward, but alas the guys that 
did the bulk of the content management are history.

> (3) Legal - IP, Copyright and Licensing
> We will need to make sure that we have clear documentation of the copyright and license
for the content on the wiki and user forums. We'll need to make sure that the Migration does
not infringe on anyone's copyright and/or inadvertently change any licenses. If we are clear
about what we are planning for these sites on an openoffice.org domain then it *might* be
possible to take everything.

The forum content was all under the Sun then Oracle terms which roughly 
mirror CCA but with OOo retaining full rights to use.  All bar a few 
dozen of the wiki pages likewise.  

*** 2011-08-01-07:42 Terry Ellison - Server Capacity Considerations, Confirming Operation

> [C]an you give me some Ubuntu VM specs so I can create them for the Forums
> and the Wiki -- we can then get them setup and working as a test and ready for a final
dump/load
> when we switch over, in the meantime we can check the loads of the VM hosts etc to be
sure
> all will be ok.
>
> Just amount of RAM and disk  space required should be all I need - and 1 or 2 cpu , then
I'll get an
> Ubuntu VM up for each. We try and stick to LTS releases so will get it to 10.04.3 LTS
version unless
> you have a reason we should use a later Ubuntu release.

I track Ubuntu current for my laptop and home server, and use Ubuntu LTS 
by preference for my LAMP VMs.  I use a 2 disk split with a common 
immutable system image and an app-specific /var (with callback hooks in 
the startup so the app can tailor the system).  I've just rebaseline my 
VMs from 10.04-1 LTS to 10.03-1 LTS.  If you guys have already developed 
an Ubuntu VM template then I can always pick up that. If not then I'll 
write up my template and make sure the licensing of my IPR / content is 
OK, so you have the option to reuse it.

I would prefer to stick with Ubuntu VMs for now because I like to keep 
local mirrors of any prod system for release development/rehearsal and 
hunting down live problems.  I currently use VirtualBox, but I can 
reinstall VMserver on my local server so that I can switch my VMs from 
VBox Guest Utils to VMware Tools.  (I like to exactly mirror any prod 
systems locally.)

Installing and getting to grips with the bowels of FreeBSD server is 
just another learning curve that I would like to avoid at the moment.  
I've already got a learning curve on compliance with your own 
infrastructure standards on Identification & Access Control, Backup, 
Logging, Intrusion Detection, Status Report, ....


*** 2011-08-01-09:53 TerryE - More on Confluence Migration

*I* have looked at [migration to Confluence] 
and it *will* be hard. I didn't say impossible.  
 ... Just look at

*  http://wiki.services.openoffice.org/wiki/Special:Statistics
*  http://wiki.services.openoffice.org/wiki/Special:Version

for the volumetrics and MediaWiki extensions used.  If you google 
"mediawiki confluence migration" then you will see that isn't a trivial 
exercise even for wiki using standard MW with no extensions: it would 
involve person-years of effort.  I have suggested ... rehosting but on a 
sustain basis for the wiki.  I can set all of this up, if agreeable to 
the project.  It doesn't need further material resources from the Apache 
team.  This would at least buy us the time to do a proper migration plan 
and resource it.

*** 2011-08-01-20:03 TerryE - Working to Have Confirmable Configurations, Trial Operation

> I'd like to get going so am going to create 2 x VMs of 50GB space and 2GB RAM each, both
> with Ubuntu 10.04.3. One for the mediawiki and one for the forums.

The wiki might struggle 
without tuning, since the current config isn't and the MediaWiki engine 
is a D/B hog unless you stick a cache such as Squid or Varnish in front 
of it. ...

I'll want to mirror 
them exactly for my dev, so can you mail me the dpkg -l listing together 
with the etc tarball and details of any not debian installs.

I'll shout if there's anything missing.  We'll need a PHP accelerator -- 
Xcache or APC, and php5-cli.

*** 2011-08-03-08:07 Terry Ellison - Reaching the Wiki Users, Legal Concerns
> [...] Is there a announcement list or some other mechanism
> to send an email to every registered wiki user?

At a technical level, it's simple to run a query dumping all of the mail 
addresses of contributors to the wiki.  I've just done a few on my local 
VM which has a snapshot of  the prod wiki as at Thursday/Fri night IIRC.

    * There are 34,969 registered users.  Of which
    * 3,675 have made contributions.  There is no need to contact those
      who haven't
    * 3,623 have registered email addresses and have made 182,677
      contributions
    * 52 have no registered email addresses and have made 153
      contributions (prob dating back to the early days when email
      registration and confirmation wasn't mandatory

It is trivial to dump this list of user / email addr / post count.

However giving this to Apache and the project making use of it is a more 
complex issue.  The server is current located in Oracle's Hamburg 
facility under German / EU legislation.  We have data protection 
legislation and Anti-Spam guidelines / legislation to bear in mind 
here.  Moving email addresses across national and organisational 
boundary might trigger these.  Also one can't send out mailshot emails 
in the EU unless the recipients have first agreed in principle to accept 
these.

What I can do is to provide this data to Andrew via the internal Oracle 
email, and let him figure out the legal / compliance issues and terms of 
use before him making it available to the project.  ...

*** 2011-08-04-01:36 Terry Ellison - Moving the WIki "As-Is" for Now

I am working with the rest of the project to 
migrate the forums and the wiki "as-is" for now.

.  As far as the wiki goes in the medium to longer term, the project may 
decide to move some material to cwiki, but this is work in progress.

*** 2011-08-04-02:05 Terry E - Registering and Authorizing Users and Their Edits

> 1) People must ask for an account; they can't self-subscribe. Nothing is
> required except a few words about who you are and why you want an
> account. Any one of several people authorised to approve or reject these
> requests can deal with them expeditiously. Very few spammers, in my
> experience, take the trouble to actually request accounts.
We need to implement this in a way which sits within MediaWiki 
functionality and complies with the goals.

One way would be

    * to allow the normal self-registration and optional email address
      with email verification
    * and have a new wiki role, say "contributor" (or is this
      contributer in US-speak?).
    * guest have no write access
    * registered users can write to User and User_talk namespaces but to
      no others
    * registered users can request to become a Contributor, but the must
      have completed their User page, verified their email address and
      confirmed that all future edits to the Main or Talk namespaces are
      made under licence (CCA AL2 or whatever we decide.
    * the granting of Contributor is done by the bureaucrats.
    * The Main and Talk pages contain "reference" content.
    * There is a standard disclaimer that user/user talk is user content
      is user content
    * We would still need main and user namespace guidelines TOUs.

This might seem a little convolved, but this can be configured with std 
MW/extension functionality.

> 2) Alternatively, or in addition, the first X edits/ contributions/
> whatever are moderated by a group of people, any one of whom can approve
> or reject the items. After X acceptable contributions, the person is
> then allowed to edit the wiki without further supervision [ ... ]
We could add another committer layer so that contributer (but not 
committer) edits are moderated

However, I suspect that a trust-but-verify attitude is easier for 
everyone.  When we catch contributers deliberately abusing the rules, 
then we can always back out their changes and remove contributer 
status.  This is similar to our forum model and works well there.

*** 2011-08-04-04:30 TerryE - Wiki Extension to Require Review-then-Commit

> You probably know more about this than I do, but my understanding is that the current
OOo wiki has an extension installed that does what I was suggesting in option 2, but the extension
has not been implemented.  See:
> http://www.mediawiki.org/wiki/Extension:FlaggedRevs and specifically:
> http://www.mediawiki.org/wiki/Extension:FlaggedRevs#Automatic_user_promotion
>
Yes, you are correct.  This extension can do this and more, but with 
a grey issue like this I feel that a DL based dialogue isn't the best 
way to work out what to do here.  Better we work up a position 
paper/page within the OOOUSERS cwiki laying down the options, their pros 
and cons and then agree a consensus or vote either on the paper itself.  
Use the DL to note the consensus and get wider feedback.

What concerns me is the moderation load involved with such an active 
intervention of review-before-publish.  Perhaps others with moderator 
experience might care to comment?

My worry is that review-before-publish also ignores the reality of how 
people edit wikis.  In general they don't prepare and proof draft 
offline then paste their best and final into the article.  Most do it 
section by section or end up correcting / rewording when they see the 
final version, so one logical edit can comprise half a dozen posts.  I 
am not sure how this would work if you've got to wait for approval 
before the next edit.

*** 2011-08-05-15:18 Terry Ellison - Migration Systems Setting Up

> Gav, the Admin from the ASF has setup the VM for the MediaWiki. It's a 
> Ubuntu 10.4.3 LTS VM. At the Moment Gav and I has admin access to this 
> machine. First we have to install all the needed software, Then we 
> will make a test migration with a older Dump, then we can make 
> testings and finaly the final migration.
>
> Greetings Raphael

Raphael, great news.   I've my dev shadow VM up as well, and just logged 
onto minotaur for the first time with my new apache ID.  I'll ping the 
config detail over to you and Gavin and take it from there.  Regards Terry


*** 2011-08-07-10:49 TerryE - Questions About the Wiki and Customizing Pages

> It should be possible to alter the login page to show any important 
> information we want to share about the migrated site, correct? With 
> possibly a survey?
>
> Also, what is the purpose of the Bots group?
 The WikiEditor group is the default group for a MediaWiki extension, 
 FlaggedRevisions, which facilitates some of the control functionality 
 that Rob, you and other have been asking for.  If you look at 
 Clayton's status page, this extension isn't yet put to use.  A job for 
 post transition, I think. But that's why Clayton is the only current 
 member.

 Yes, the Main_Page is a restricted page, but anyone can take a copy 
 and work on it in their User page hierarchy.  When we have a consensus 
 that it's OK then I can move it back into main.

 The bots group is for a  function supported by MediaWiki  where it 
 provides an [XMLRPC] to all batch process to  work on the wiki.  In this 
 case Clayton and co used the PyWikipediabot (I think) to carry out 
 bulk transactions. This needs an account with special privileges.

*** 2011-08-07-15:38 Terry Ellison - WORK ITEMS AND ISSUES - FIRST CUT

I've just finished a 1st cut of outstanding tasks and issues for the Wiki.

See 
https://cwiki.apache.org/confluence/display/OOOUSERS/Community+Wiki+Services 


Comments gratefully received on this DL and/or on the page itself.

*** 2011-08-08-02:22 Terry Ellison - Derisking Migration/Staging - Separating Infrastructure/Platform
and Content

Thank-you all for your replies so far.  I will work through and respond 
to individual points in thread at the appropriate branch, but here I 
just wanted to propose that we divide the Wiki migration into two 
separate but task areas:

    * Migration of the infrastructure service.  What drives this is if
      Oracle decides, say, that the current hardware is being turned off
      on the XXth of Aug, then we need to move the service to a box in
      Apache.org.  The issues here are largely technical heavily involve
      the infrastructure team.  We need to be able to move quickly on
      this and few of the DL contributors have strong opinions here. 
      They just want the 'magic to happen'.  Yes, there are a few
      decisions to be made, but the questions are a little esoteric for
      most DL members.  I offer to continue to lead this work area.

    * Migration of the wiki content / policy.  Here we have a range of
      widely and strongly held views and the DL seems to be acting as a
      debating shop without convergence to consensus on some points. 
      This debate could still be continuing in 3 months time as far as I
      can see.  I don't want it to get in the way of infrastructure
      migration -- except when /absolutely/ necessary.  It also makes
      sense to break this task area further if this means that we can
      make progress in some areas, even if stalled in others.

but from an infrastructure 
viewpoint I don't really care this branding process can be encapsulated 
and detached.

    * We make the pre-prod wiki available to the "branding team"
    * The branding team agree and implement the branding changes that
      the project wants to have in place for cut-over day.  In practice
      this could involve the creation or modification of content in the
      Main, File, MediaWiki, OpenOffice.org Wiki and Template Namespaces
    * I can use standard MW audit functionality to capture a list of all
      such pages, and then standard MW export functionality to create an
      XML.GZ dump of this work.
    * We then import the "live" wiki as part of cut-over.
    * I then reimport the above XML.GZ to reapply the branding changes
      to the migrated wiki.

... [M]y suggestion is to keep the infrastructure and content issues 
separate.  I can lead on the first.  We need someone who can lead on the 
second. His or her strengths should be that he/she has experience of 
delivery -- that is can make things happen -- and has a good working 
knowledge of more advance MW features such as templates, etc.  I would 
think that either Drew or TJ would be well qualified, but that is only 
my personal suggestion.

*** 2011-08-08-01:41 Terry Ellison - Getting Help on the Content Side

> I can submit an iCLA if it would help (it's on my WIGATI list).

[W]e need someone has sound MediaWiki skills to lead on doing these 
[content]  I've suggest that you are 
qualified.  I don't know if anyone on the DL other than you or I knows 
enough about MW content editing, so could you consider my suggestion?

*** 2011-08-08-02:39 TerryE - More on Content Migration

> I assuming that under the name "branding changes" we include:
>
> 1) OpenOffice.org ->  Apache OpenOffice.org and associated logo changes
>
> 2) Removal of Oracle logos and name
>
> 3) Replacement of privacy policy and disclaimer
>
> 4) Update of page edit license text to require Apache 2.0 for new contributions
Yes, good points, though such content changes are functionally separate 
from the (infrastructure) migration.  I have suggested in a separate 
branch that we set up separate task area with its own leader to work 
these issues.

*** 2011-08-08-03:16 Terry Ellison - Wanting to Walk through It At Home

To be honest I'm not comfortable with any solution unless 
I've used myself, ... This {??] option is simple to 
implement and try in pre-prod, unlike a migration to PostgreSQL which 
will involve quite a bit of work.

(An example of where I am comfortable is with the use of PHP-APC).

What I had hoped to get feedback/from the wider ooo-dev DL/ was on the 
least technical risk, but which also had the business impact, and that 
is to stop the service for ~30 mins a day to do the backup.

Is this acceptable?  Historically, the lowest transaction rates occur at 
~04:00 UTC, so this is when I'd do it if we go this way.

*** 2011-08-08-03:33 Terry Ellison - Risks of Changing DBMS along with Migration

There are two factors against an early move to PostgreSQL:  (i) The 
MediaWiki cavaets on its use (here 
<http://www.mediawiki.org/wiki/Manual:PostgreSQL> and as all experienced 
Wikipedian's do look at the associated talk page here 
<http://www.mediawiki.org/wiki/Manual_talk:PostgreSQL>; this has a 
couple of interesting references which cookbook the conversion).  (ii) 
The extra work involved. This is not only the D/B migration, but also a 
MW version upgrade 15.1 -> 17.

To be honest I am uncomfortable doing this as part of this immediate 
"continuity of service" migration.  My suggestion is that if the wiki 
looks as if it is going to have a long term place within  the project 
then we should revisit this as part of an in-service improvement program 
in, say, 6-12 months.

*** 2011-08-08-04:14 Terry Ellison - Migrating Static Web Projects to Wiki

> I strongly suggest that all the native 
> language projects front-facing pages be migrated to the "new" wiki.
[ ... ]
> This would of course, give them URLS of the form
> somewikiname.openoffice.org/"nl_code"
> rather than "nl_code".openoffice.org
[ ... ]
> Naturally, we should confirm acceptance with them for this.

... I see this as part of the content migration / transformation 
task area.  What I want to go is to get the current content over with 
loss of content, formatting or change history.

... I assume by your refrence to the "new" wiki 
... you mean the ooo-dev cwiki.  As we've discussed before, the current 
migration from MediaWiki to Confluence can be highly lossy unless there 
is per-page content editor intervention: we [lose] all  change history, 
some content formatting (1) and some even content (2).  It therefore 
makes sense, even if we migrate the publicly viewed copy to cwiki, to 
keep a master copy of the old content online in ... this wiki, albeit 
moving the page into an "Archive" namespace which is private and 
read-only to committers.

*** 2011-08-08-05:03 Terry Ellison - Migration Isn't Just Moving a System Dump

> If Oracle decides, say, that the current hardware is being turned off
> on the XXth of Aug, then we need to grab a backup of the content to a 
> box in Apache.org.

It's a big more complicated than just grabbing the content :)  I already 
routine off-site the current forum and wiki system content for dev 
support and [Disaster Recovery], but that's within current working 
practices that date back to Sun days.  It's putting content to new use 
with a.o that requires clearing the hurdles.

*** 2011-08-09-02:16 Terry Ellison - LOOKING AT WORK-ITEM ISSUES FEEDBACK, STEPS BEING TAKEN

[Re 2011-08-07-15:38, https://cwiki.apache.org/confluence/display/OOOUSERS/Community+Wiki+Services]

Thanks to everyone for this constructive feedback.  I hope that I have 
given replies in thread fork where you needed then, but I will up 
integrate them and update the document and move forward to complete task 
6 in the next couple of days.  There are two further points that I would 
like to raise for comment / feedback.

(1) This bulk of this page content relates to technical details and 
issues, rather than content / branding / licensing, etc..  As I 
mentioned in a reply to RobW, I would like to decouple the 
infrastructure aspects from the content-relate aspects as much as 
practical to keep the momentum on the platform move.  I therefore 
propose to move the bulk of the infrastructure content into a 
supplementary page which focuses on this content.

(2) I will be making the snapshot version accessible to the project, 
however:

    * I will be overwriting all passwords (except mine and Drew's) thus
      disabling existing account use.  I will also replace all email
      addresses by dummies to prevent existing users resetting their
      passwords, and user page-watch information to stop bad emails
      being generated.
    * Anyone who one who wishes to work on the wiki will need to create
      a new account and apply to Drew or myself for temporary elevated
      privileges if needed.  Just remember that new content will be
      blown away at cut-over, so individual new users must use the
      standard export functionality prior to cut-over to save it if they
      want to restore it post live-transfer.
    * As step one removes the private data elements that could be viewed
      as falling within the purview of EU data protection legislation,
      this means that we could in principle make a copy of this D/B to
      any committers for data analysis. All they need is MySQL installed
      and 3Gb space the D/B.  The data model is pretty straight forward
      :-) (*)

(*) a quick example here is that the edit profile is a classic near 
log-lin relationship: 33% of edits where made by 10 editors, 73% by 100, 
90% by 315.

*** 2011-08-10 15:12 Terry Ellison TEST/PRE-PRODUCTION WIKI INSTANCE PROGRESS

Just a quick update on progress on the wiki instance.  This is now up in 
preprod mode  on the subdomain ooo-wiki.  This is based on a snapshot 
taken at 4am yesterday morning.  The server needs further tuning, but 
functionally its there.  We do have some outstanding code issues (due to 
a change in the functionality of the PHP routine*call_user_func* between 
Version 5.3 and 5.2 which is not backwards compatible.  I'll sort this 
out tomorrow).

Also note as per my previous email, all existing account have had there 
passwords mangled, so access is guest-mode only at the moment. Feel free 
to have a look, but updates and no service commitments yet.

*** 2011-08-11-01:18 Terry Ellison - Rig for Silent Running (robots.txt)

>> <http://ooo-wiki.apache.org>
>>
> Hmmm... if this is just a temporary URL, I wonder if we should try to
> exclude search engine indexing via robots.txt ?
>
> Otherwise, now that the URL is known, we're going to get spidered but
> then all those links will soon be dead.
> <snip>

Thanks Rob for pointing this out, and for Raphael and Gavin fixing it.  
This was incidentally on my bucket list for today.  I didn't think it 
that urgent as it would take a day or so for Google et al to acquire the 
site, but you correct.  Better sooner.

*** 2011-08-11-01:38 Terry Ellison, BREAKAGE IN SEPARATING WIKI FROM OO.O
> I found one bug:
> On the wiki left side lowest frame where store links for other 
> languages main pages,
> the links points to http://wiki.services.openoffice.org, not to the 
> http://ooo-wiki.apache.org/.

You are correct, and in fact there are a lot of references to 
*.OpenOffice.org in the content.  However, there isn't a production 
instance and there little point in changing these until (1) we agree 
what such "dirty" references are going to be changed to, and (2) we cut 
over for real.  Remember that any changes that we made on this instance 
will be lost when we bring over the live [database].

2011-08-12-04:53 Terry Ellison, OO.O WIKI ERROR LOGS BUMP

As part of my takeover of SysAdmin on wiki.services.openoffice.org, I've 
been reviewing the error logs and noticed that last week there's been 
quite a rise in reported errors by clients trying to enumerate the wiki 
incorrectly.  This may just be a coincidence and nothing to do with our 
work, but if anyone on this DL is trying to suck material down, there 
are right ways to do this.  I also understand the MediaWiki webAPI if 
any of you want to write a bot to do this. ... Please contact me to discuss.

*** 2011-08-23-20:30 Terry Ellison, SPEEDING UP WIKI ACCESS, REDUCING LOAD

I've now finished the upgrade to add the Apache Traffic Server front end 
to the community MediaWiki service at http://ooo-wiki.apache.org/ and 
the service is back online.

We need to do some further tuning of the system cache optimisation, but 
even with the first-cut settings that I prepared on my own test-bed VM, 
the system already looks as if it it is hitting the performance targets 
needed to sustain a full production transaction rate.  It certainly 
feels extremely snappy compared to the existing OOo community wiki or 
the Apache cwiki, and the Google pagespeed benchmarks are significantly 
better than both of these systems. So:

    * We are good to go for production migration of the community wiki.
    * We are the first Apache project adopter of another Apache project,
      Apache Traffic Server (ATS)
    * We have laid the foundation for an ATS template for MediaWiki
      hosting that can be used to promote the use of ATS for the wider
      MediaWiki systems community.

It's been a hard few days work, and still some finishing to do, but my 
thanks to the Infrstructure and ATS guys that have supported me in 
making this happen.

*** 2011-08-25-08:59 Terry Ellison, REQUEST FOR BASELINE AGREEMENTS TO MOVE AHEAD

I can't execute any plan without a baseline requirement and set of 
assumptions, so what this note attempts is to lay down such a set, and 
the decisions that need to be made to go forward.  So PLEASE, I don't 
want any flames about my use of DECISION below.  What I simply mean is 
the if the PPMC as a body accepts these, then I will try my best to move 
this work forward.  Of course you are free to challenge / change any of 
this if that is a PPMC voted decision, but in this case I need to move 
into a different mode; to suspend work and stop the clock until we have 
an PPMC-endorsed baseline to replan on. ...


*    The infrastructure stack is base on a standard Ubuntu server LAMP 
stack as at current LTS (Ubuntu 10.04-3 LTS) which included PHP 5.3.2

*    The prod wiki is v1.15.1 that at an N-3 major release level (that's 
30 months old: two major and 10 minor revisions behind the current 
supported).  This also runs on PHP 5.2.0.

*    We need an reverse-proxy HTTP cache for performance reasons on the 
wiki.  One of the four market leaders in this niche is another Apache 
project: Apache Traffic Server (ATS).  It makes sense to stay "in-house" 
here for both support and referenceability reasons

* *DECISION*: Adopt ATS v3.0.1 as the HTTP cache for the wiki.  (BTW, 
this work has been done and the product is excellent).

The PHP 5.3 introduced extra checking to remove an area of tolerance the 
PHP 5.x<3 allowed.  This was to do with when and how parameters can be 
passed by reference under curtain circumstances.  So moving a code base 
from 5.2 to 5.3 involved a lot of work identifying and eliminating this 
mis-codings.  This was done by the MediaWiki team in MW v1.16.  I had 
planned to move to MW v1.15.5 (the last stable 1.15.x) as our baseline 
and I've done this work integrating it with Apache Traffic Server (ATS) 
and our LAMP stack.  This is stable and performant enough to show that 
we are good.  However, I have only identified and bug-fixed the main 
path 5.2->5.3 coding issues.  During my testing I have subsequently 
discovered others and there are undoubtedly more to find.  I've also 
discussed this with the MW devs on the MW IRC channel.  Given this, the 
consensus in the @infra team (me included) is that we should bite the 
bullet now and move to current MW 1.17.0 even given the extra work.  
There are some performance risks associated with MW 1.17.0 which we need 
to mitigate.  However, given that we've already got a complete 
LAMP+ATS+MV in an ESX hosted VM performing like a dingbat, we really 
only face the 1.15.5 -> 1.17.0 issues in this step.

* *DECISION*: Upgrade the ooo-wiki MediaWiki(MV) + all extensions to MW 
v1.17.0.

* *DECISION*: I have agreed with infrastructure that we will keep 1 core 
on "standby" so we can up the VM to a 2-core VM if we are seeing 
unacceptable performance problems with one-core.

* *DECISION*: We will cut over the wiki and the forums with the content 
as-is and implement branding and access control changes within the a.o 
infrastructure when volunteers come on-stream to resource this.  This is 
the standard "transfer then clean-up" approach adopted when a migration 
is time critical.

*CUT-OVER*

There are two facets to cut-over: content move and DNS-based IP 
reassignment.  Clearly we need to freeze update access to the services 
prior to start of content move and continue update-freeze on the legacy 
service.  Bringing the content across involves a backup, copy restore 
which can be rehearsed and scripted, but in the case of the wiki, this 
will be a few hours even if fully automated.

*    There are many way "to skin the cat" of the migration process.  All 
will involve some service loss, but the complexity of the rehearsal and 
planning come explode as we reduce this outage to a zero.  Complex plans 
can also go wrong so my instinct is to keep it simple: halt the service 
at a pre-notified time, transfer and start new service at  a 
pre-notified time.

* *DECISION*: Halt the wiki service for a notified (24hr) window during 
cutover.  The migration uses fixed IPs, so  DNP IP reassignment is 
co-incident with service stop.

* *GOAL*: Cut over [wiki] within 14 days from today.  Date TBD by PM.  I 
can do the content move.

*     We have some further caching tweaks on the interaction of the 
MediaWiki [application] with the ATS HTTP reverse proxy cache, but these 
are probably nice-to-have than essential.  More to the point these need 
to be done on a system will production load patterns.

* *DECISION*. We will defer such tuning until post go-live.

*** END OF THE WIKI NOTES ***


Mime
View raw message