incubator-ooo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dennis E. Hamilton" <dennis.hamil...@acm.org>
Subject [forums] Migration - A TerryE Clipping Collection [NOT QUITE SO LONG]
Date Wed, 07 Sep 2011 03:15:20 GMT
This is another clipping stream compiled in the same manner as the one on Wiki Migration.
I'm breaking a thread here, again. There is no need to comment on any of this, it is for reference
and mining for things to not overlook, collate onto the wiki, etc. 

The selections are mine, the list is long, and it probably works better distilled onto a wiki.
 This is the collection phase.

I am staying as close to the architectural and technical considerations that I can, and not
follow into areas where there were policy spats of one kind or another.  I thought I might
have to look at others posts too, but generally, I could find enough in what Terry was responding
to provide continuity.

That does not mean there are no policy topics, but only to the extent there is an impact on
technical direction and resource, the kinds of risk-management issues that Terry raises.


NOTE: Terry also provided OOOUSER Material


*** 2011-07-31-13:29 all times PDT (UTC-0700), Terry Ellison, Request for Planning Items

> We have areas on the Community Wiki for planning transition. [1] [2]
>
> [1] https://cwiki.apache.org/confluence/display/OOOUSERS/Site-PPMC-Plan
> [2] https://cwiki.apache.org/confluence/display/OOOUSERS/Transition+Planning
>
> Please create a page on the wiki under "Transition Planning" with these and further details.
We'll then engage Infrastructure and get forward traction on finding the home for user.services.openoffice.org.

That's what I plan to do.  If you look at my user page on the (old) OOo 
wiki, you'll see that I am used to working this way.



*** 2011-07-31-13:53 Terry E - Concerns for Information Loss, Incompatibilities depending
on Migration Cases

Migrating this knowledge to another application which does not preserve 
logical data models (viz any other application) would be a project 
beyond the capability of this team.

> On the customizations, are they purely UI, or do we have functional
> patches as well?

See 
http://wiki.services.openoffice.org/wiki/User:TerryE/phpBB3.0.7_Upgrade/Closed_Issues_and_Actions#Forum_Customisation

The alternative to getting [phpBB] support within Apache.org is 
to discard the knowledge in 300,000 posts across the 9 forums.

*** 2011-07-31-15:07 Terry Ellison, Basic Platform Requirements for Servers

Both the OOo forums and the OOo wiki can run on a standard LAMP stack.  
I've pretty much optimised the forums for a LAMP stack; they need a 
one-core equivalent CPU load and with 4Gb RAM enough is memory-cached to 
have a tiny I/O load.

*** 2011-07-31-16:02 TerryE, Simplicity of the Forum Systems, Content licensing considerations

The Forums are simple.  All 9 language forums use a common code base 
which includes a single style template which is supported by the phpBB 
team.  There are common header and footer templates within this which we 
have customised for OOo.  I imagine the move from Oracle to Apache.org 
will be of similar complexity to that of Sun to Oracle unless the Apache 
team require us to move from this phpBB "Prosilver" look and feel, then 
it would involve a lot of work.

> (3) Legal - IP, Copyright and Licensing
> We will need to make sure that we have clear documentation of the copyright and license
for the content on the wiki and user forums. We'll need to make sure that the Migration does
not infringe on anyone's copyright and/or inadvertently change any licenses. If we are clear
about what we are planning for these sites on an openoffice.org domain then it *might* be
possible to take everything.

The forum content was all under the Sun then Oracle terms which roughly 
mirror CCA but with OOo retaining full rights to use.

*** 2011-08-01-07:42 Terry Ellison, Configuration, Testing, and Staging Considerations
[See Under the Wiki Customization/Migration Clippings - configuration, testing, and staging
considerations]

*** 2011-08-01-20:03 TerryE - Selecting Platform Configurations

> I'd like to get going so am going to create 2 x VMs of 50GB space and 2GB RAM each, both
> with Ubuntu 10.04.3. One for the mediawiki and one for the forums.

The forums will comfortably run in this.

*** 2011-08-04-01:36 Terry Ellison, Presumed "As-Is" Approach

I am working with the rest of the project to 
migrate the forums and the wiki "as-is" for now.

I don't know of any plans to change the forum operating model materially

*** 2011-08-05-05:48 Terry Ellison, Help Adding More Forums for Native Languages

On the admin side, we've set up 9 forums in a very scalable way and done 
a lot to make them zero admin.  Adding another for another Apache 
project if wanted is pretty trivial.  There are also probably a dozen 
people with in-depth moderation experience who might be willing to 
provide consultancy [on operating a forum].

*** 2011-08-11-08:15 Terry Ellison, TEST INSTANCE STATUS

Just a quick update on progress on the forums instance.  This is now up 
in preprod mode  on the subdomain ooo-forums.  This is based on a 
snapshot taken at 4am Tuesday [2011-08-09] morning.  The server needs further tuning, 
but functionally its there.  Also note as per my previous email, all 
existing account have had there passwords mangled, so access is 
guest-mode only at the moment. Feel free to have a look, but updates and 
no service commitments yet.
PS.  I'll do the robots.txt now

*** 2011-08-24-11:58 Terry Ellison, Anticipating Forum Traffic

On our forums alone we have been averaging over 4 times this 
rate [81 posts/day] of posts for years, and because we actively encourage users  to 
pick up related threads, we have 5.3 posts per topic. And the discounts 
the 10x plus Google driven browsing of previous answers.   Unanswered 
posts run at 6.2% which is a bit higher at we would like, but many of 
these "unanswered" posts are announcements and volunteer-submitted 
examples so the true rate is probably under 5%.

*** 2011-08-25-08:59 Terry Ellison, REQUEST FOR BASELINE DECISIONS FOR GO-FORWARD

I can't execute any plan without a baseline requirement and set of 
assumptions, so what this note attempts is to lay down such a set, and 
the decisions that need to be made to go forward.  So PLEASE, I don't 
want any flames about my use of DECISION below.  What I simply mean is 
the if the PPMC as a body accepts these, then I will try my best to move 
this work forward.  Of course you are free to challenge / change any of 
this if that is a PPMC voted decision, but in this case I need to move 
into a different mode; to suspend work and stop the clock until we have 
an PPMC-endorsed baseline to replan on. ...


*    The infrastructure stack is base on a standard Ubuntu server LAMP 
stack as at current LTS (Ubuntu 10.04-3 LTS) which included PHP 5.3.2

*    The forums are stable, but at an N-1 release level. (phpBB 3.0.8 
vs. 3.0.9).

* *DECISION*: Upgrade the ooo-forums phpBB app + customisations to 3.0.9 
before go-live. (Based on my last 5 upgrades, this 1-2 days work, the 
main part being the regression of a 1K line customisation patch when we 
rebaseline the package from 3.0.8 -> 3.0.9)

* *DECISION*: We will cut over the wiki and the forums with the content 
as-is and implement branding and access control changes within the a.o 
infrastructure when volunteers come on-stream to resource this.  This is 
the standard "transfer then clean-up" approach adopted when a migration 
is time critical.

*PRIORITISATION*

One the one hand the forums involve a lot less work and technical risk.  
On the other they are arguably also used more than the wiki at the 
moment, and the post rates are a LOT higher.  Because I am a single 
resource constraint, I can't do both at the same time.  This one is 
toss-a-coin, but my instinct is to get the one that we can do quickly 
done.  But if there is a strong consensus to the swap then I can do it.

* *DECISION*: The priority is to work the forums over the wiki.  We cut 
the forums over first.

*CUT-OVER*

There are two facets to cut-over: content move and DNS-based IP 
reassignment.  Clearly we need to freeze update access to the services 
prior to start of content move and continue update-freeze on the legacy 
service.  Bringing the content across involves a backup, copy restore 
which can be rehearsed and scripted, ...

*    There are many way "to skin the cat" of the migration process.  All 
will involve some service loss, but the complexity of the rehearsal and 
planning come explode as we reduce this outage to a zero.  Complex plans 
can also go wrong so my instinct is to keep it simple: halt the service 
at a pre-notified time, transfer and start new service at  a 
pre-notified time.

* *DECISION*: Halt the forum service for a notified (24hr) window during 
cutover.  The migration uses fixed IPs, so  DNP IP reassignment is 
co-incident with service stop.

* *GOAL*: Cut over forums within 7 days from today.  Date TBD by PM.  I 
can do the content move.

*** 2011-09-01-04:34 Terry Ellison, BUILDING/MIGRATING THE FORUMS TO TEST INSTANCES

I moved my "site" directory tree on ooo-forums.apache.org last night.  
(This is the ~80 files that need to be put under svn, and that enable 
the site-customised forum configuration to be rebuild by one of the 
scripts in site/scripts and the phpBB tarballs that it pulls from the 
phpBB repository.)  We still have to finalise where this repository 
location will be.

It then took me ~40 mins to do a scripted rebuild bringing the forums up 
to phpBB v3.0.9 (current).  This included pulling the latest forum 
backup from user.services.openoffice.org and restoring this into the 
local application. So if you look at http://ooo-forums.apache.org/ today 
then you will now see that it's running the latest version of phpBB with 
a clone of yesterday's live forum content.

I still have some regression tests to do and some temporary D/B tweaks 
for this version (e.g. the forums are configured as 
user.services.openoffice.org and not ooo-forums.apache.org and this 
mucks up cookies), but it broadly looks as if we will be good to do.

The work for the live cut-over will take about 30mins, with most of this 
is network transfer and loading and reanalyzing databases.  I will 
coordinate a timeline with Andrew and come back with an update.  
Unfortunately, we will still need a nominal 24hr service outage for 
cut-over because of DNS propagation ripple.

*** 2011-09-05-08:10 Terry Ellison, INCLUSION OF THE JAPANESE FORUM (AND OTHERS)

You can see the last migration copy of the JA 
forum at

    http://ooo-forums.apache.org/ja/forum/index.php

This includes the upgrade to v3.0.9. and the snapshot was taken at UTC 
11:00 today (that is 5 hrs ago).

  *** END OF THE FORUM NOTES


Mime
View raw message