incubator-ooo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Schaefer <joe_schae...@yahoo.com>
Subject Re: suggested CMS workflows for ooo-site
Date Wed, 04 Jan 2012 19:06:20 GMT
FWIW I opened a thread on dev@svn about the time it takes
for svn to commit and merge large changesets.  If there's
nothing more they can do to improve the situation, I have
a hunch that migrating the cms to a machine with SSD's running
l2arc cache for zfs will help a lot.

We have a machine lying around that would suit that purpose,
just haven't had the time to bring it up and migrate over to it.
Eventually tho I'll get the free tuits to make it happen.



----- Original Message -----
> From: Joe Schaefer <joe_schaefer@yahoo.com>
> To: "ooo-dev@incubator.apache.org" <ooo-dev@incubator.apache.org>
> Cc: 
> Sent: Wednesday, January 4, 2012 12:52 PM
> Subject: Re: suggested CMS workflows for ooo-site 
> 
> ----- Original Message -----
> 
>>  From: Dave Fisher <dave2wave@comcast.net>
>>  To: ooo-dev@incubator.apache.org
>>  Cc: 
>>  Sent: Wednesday, January 4, 2012 12:40 PM
>>  Subject: Re: suggested CMS workflows for ooo-site 
>> 
>> 
>>  On Jan 4, 2012, at 9:29 AM, Joe Schaefer wrote:
>> 
>>>   Also Dave get in the habit of checking buildbot for the
>>>   build status of sledgehammer commits instead of waiting
>>>   for svnmailer to figure out what to do with the massive
>>>   diff it's trying to make sense of.  The url is 
>>> 
>>> 
>>>   http://ci.apache.org/builders/ooo-site-site-staging
>> 
>>  I do do that, but tend to wait for the email anyway. If there is no reason 
> to 
>>  wait that will save time.
> 
> Buildbot performs the commit back as the final step in the build,
> so if buildbot thinks the build has completed successfully, you
> do not need to wait for svnmailer to send out a notice to that effect.
> 
> My experience is that the turnaround between sledgehammer commits
> and eventual publication is about 1 hour: ~20 min for each step
> along the way, all because of svn committing or merging huge volumes
> of data.
> 
>> 
>>> 
>>>   You may want to turn this thread into some basic cms
>>>   usage documentation on the incubator/openofficeorg site.
>> 
>>  Yes. I think it is time to rework what is on the incubator site.
>> 
>>  See incubator/openofficeorg/website-local.mdtext.
>> 
>>  But not today. I'll let you know when and we can review it.
> 
> Sounds like a plan.
> 
>> 
>>  Regards,
>>  Dave
>> 
>> 
>>> 
>>> 
>>> 
>>>   ----- Original Message -----
>>>>   From: Joe Schaefer <joe_schaefer@yahoo.com>
>>>>   To: "ooo-dev@incubator.apache.org" 
>>  <ooo-dev@incubator.apache.org>
>>>>   Cc: 
>>>>   Sent: Wednesday, January 4, 2012 11:51 AM
>>>>   Subject: Re: suggested CMS workflows for ooo-site 
>>>> 
>>>>   You can always mimic exactly what the staging site
>>>>   does with its builds, namely target the build to
>>>>   a checkout of
>>>> 
>>>>   https://svn.apache.org/repos/infra/websites/staging/ooo-site/trunk
>>>> 
>>>>   When the build finishes you can just svn diff that
>>>>   tree to see what you actually changed, but *please*
>>>>   do not commit back those changes yourself.  You
>>>>   can nuke all your changes to that tree with svn revert -R
>>>>   so it's handy for testing different types of sledgehammers.
>>>> 
>>>> 
>>>>   HTH
>>>> 
>>>>>   ________________________________
>>>>>   From: Dave Fisher <dave2wave@comcast.net>
>>>>>   To: ooo-dev@incubator.apache.org 
>>>>>   Sent: Wednesday, January 4, 2012 11:46 AM
>>>>>   Subject: Re: suggested CMS workflows for ooo-site 
>>>>> 
>>>>> 
>>>>>   On Jan 4, 2012, at 8:28 AM, Joe Schaefer wrote:
>>>>> 
>>>>>>   Given that the size of ooo-site is around 9GB, there
>>>>>>   are some unique challenges here in dealing with the CMS.
>>>>>>   For the most part tho, the typical workflow of editing
>>>>>>   a few pages on the site, committing them, and publishing
>>>>>>   them can all be done reasonably effectively using the CMS
>>>>>>   website.
>>>>>> 
>>>>>>   OTOH, people who need to monkey with templates/** or 
> lib/**
>>>>>>   files will trigger full site builds and their changes may
>>>>>>   materially impact every file on the site.  While I've 
> now
>>>>>>   reduced the build time to around 4 minutes, the bottleneck
>>>>>>   now remains squarely in the time it takes svn to commit 
> back
>>>>>>   those changes and to deal with merging those changes 
> during
>>>>>>   publication requests.
>>>>> 
>>>>>   Thanks for your improvements. 
>>>>> 
>>>>>> 
>>>>>>   In those circumstances I strongly advise you to use the
>>>>>>   publish.pl script on people.apache.org to review and if
>>>>>>   ok publish your changes.  This will eliminate the chances
>>>>>>   that your browser times out a direct publish request to 
> the
>>>>>>   CMS site, which is a real hassle given that it takes ~15
>>>>>>   minutes for a largeish publish request to be processed.
>>>>> 
>>>>>   I always use publish.pl when I use my sledgehammer ;-)
>>>>> 
>>>>>   I usually test my changes with local build_site.pl or 
>>  build_file.pl.
>>>>> 
>>>>>   My observation is that the biggest bottleneck is more in the 
>>  creation of the 
>>>>   email reports. Particularly after publish.pl returns.
>>>>> 
>>>>>> 
>>>>>>   In the near future we will be upgrading svn to 1.7 on the 
> CMS
>>>>>>   server which will bring in better performance along with
>>>>>>   full support for deletions via svn, but I don't expect 
> the
>>>>>>   performance changes to significantly alter the workflow 
> I'm
>>>>>>   recommending here.
>>>>>> 
>>>>>>   And please for the sake of others who want to work on 
> minor
>>>>>>   changes to the site, don't make a sledgehammer type 
> commit
>>>>>>   without following up with an eventual publish request, 
> because
>>>>>>   publish requests are an all-or-nothing type deal.  That 
> means
>>>>>>   a sledgehammer commit will cause unreasonable delays for 
> people
>>>>>>   who are trying to publish minor changes to the site, until
>>>>>>   the person who did the sledgehammer commit follows thru 
> and
>>>>>>   publishes everything.
>>>>> 
>>>>>   I would recommend that larger template and skeleton changes 
> with 
>>  the whole 
>>>>   ooo-site are done locally and fully tested before committing to 
> svn..
>>>>> 
>>>>>   Do you have any recommendations for comparing a locally built 
> site 
>>  with 
>>>>   current production in order to understand how big a sledgehammer 
> is 
>>  being built?
>>>>> 
>>>>>   Regards,
>>>>>   Dave
>>>>> 
>>>>>> 
>>>>>> 
>>>>>>   HTH
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>> 
>> 
> 

Mime
View raw message