From ooo-dev-return-11553-apmail-incubator-ooo-dev-archive=incubator.apache.org@incubator.apache.org Wed Jan 4 19:06:49 2012 Return-Path: X-Original-To: apmail-incubator-ooo-dev-archive@minotaur.apache.org Delivered-To: apmail-incubator-ooo-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 616CDBBAE for ; Wed, 4 Jan 2012 19:06:49 +0000 (UTC) Received: (qmail 4391 invoked by uid 500); 4 Jan 2012 19:06:49 -0000 Delivered-To: apmail-incubator-ooo-dev-archive@incubator.apache.org Received: (qmail 4333 invoked by uid 500); 4 Jan 2012 19:06:49 -0000 Mailing-List: contact ooo-dev-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: ooo-dev@incubator.apache.org Delivered-To: mailing list ooo-dev@incubator.apache.org Received: (qmail 4319 invoked by uid 99); 4 Jan 2012 19:06:49 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 04 Jan 2012 19:06:49 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=5.0 tests=RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [98.139.52.199] (HELO nm2.bullet.mail.ac4.yahoo.com) (98.139.52.199) by apache.org (qpsmtpd/0.29) with SMTP; Wed, 04 Jan 2012 19:06:42 +0000 Received: from [98.139.52.196] by nm2.bullet.mail.ac4.yahoo.com with NNFMP; 04 Jan 2012 19:06:21 -0000 Received: from [98.139.52.170] by tm9.bullet.mail.ac4.yahoo.com with NNFMP; 04 Jan 2012 19:06:21 -0000 Received: from [127.0.0.1] by omp1053.mail.ac4.yahoo.com with NNFMP; 04 Jan 2012 19:06:21 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 244788.52873.bm@omp1053.mail.ac4.yahoo.com Received: (qmail 64861 invoked by uid 60001); 4 Jan 2012 19:06:20 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1325703980; bh=ynfdjc8aoZepmUsi6/dntmh/9mXl4/a0QcpVjQ4QOGU=; h=X-YMail-OSG:Received:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=aaiGS58k8FDtlvDQ80ekeH/abLlDK8TMNV19n8GaJGAS0WLs5WTepRG6PMBMLURzThRyw143CqXHCAkhRhFB3n+6r/xf/elTCNk8KtU6T0MZDUfTpJvenuSM7/zzCSBBIOoC4peksVdxsmbKJHT9N6kl9bJ/iFn/eHpbJFGARms= DomainKey-Signature:a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=MWIK+0FgEGtHY06rJ0p6s0cxvzixUge0w7oAqpMn8/gP0BLNlAM/sxsLAZjKQFuTD8iPEpZl2D4klB2mo6HJYkXhViPe9uSI3WMsXb/vFqnIngHkP4p9AZbiHuUsa4b4dcjb0HAsJPY6+ivDXwaSfB5bLl1SN9Znw0X5mWw3v9w=; X-YMail-OSG: 6im7aAYVM1n8ChHhWGjZ109Zz1t7BhjjvpKn1Gg97azl5nE KsEyzEUvcR7.WgUZ2ScDgzpbcIMlE58flOSV.IT05E4Ibh5UZCXl_SHbfGv4 dgCx3YmGeELiIA.zC43CfaxantPEy1uSLGdSXd3kdjwM7JinAuqZRDl_v_pn DgNOEftGroHB_M4Z3O1hcr3_LEO4B6qDUr63hd4vYAS00yCninsK5_eDAVMa Of1C.qdVn1MWLXGOU5qve6_wgTRxD0MOteTQ4KY9Usf4m8zCkV8IFf..ScUW pZhmYdIC.mm3HxRbG_HGEjXWQGgqSmlTgUWUtsTVEQ0an3SwHxK9tN3adr7w HhiRQzrECuH9nyrdlIk5GPam9jx0cu3leuLW1YyAtqeFbJ.RJ6NVo3wGCId8 RwLVTvHyFhir3NgJo7rISRS9gqYw.WMPaaEXcbkSzGdlHUFUICoheqbbkVAT UN8CeQ8vDOMHrwZkRG1iI0fdROIjWy3ErTYZo0EXCWvltK1x1VRbtlJS_4.Q miw_GHIzL.NcBnLRrDHTl5_VNaMG1I7u83CHa_SHO8FrQMYTWJLXNGvfOe9f q8XaSPtHagsFLoNzFhu7Yx7AGxcJzxTKKp2wx Received: from [99.135.28.65] by web160904.mail.bf1.yahoo.com via HTTP; Wed, 04 Jan 2012 11:06:20 PST X-Mailer: YahooMailWebService/0.8.115.331698 References: <1325694487.26057.YahooMailNeo@web160903.mail.bf1.yahoo.com> <1325695906.80326.YahooMailNeo@web160906.mail.bf1.yahoo.com> <1325698191.35878.YahooMailNeo@web160901.mail.bf1.yahoo.com> <1325699559.19044.YahooMailNeo@web160904.mail.bf1.yahoo.com> Message-ID: <1325703980.58211.YahooMailNeo@web160904.mail.bf1.yahoo.com> Date: Wed, 4 Jan 2012 11:06:20 -0800 (PST) From: Joe Schaefer Reply-To: Joe Schaefer Subject: Re: suggested CMS workflows for ooo-site To: "ooo-dev@incubator.apache.org" , Joe Schaefer In-Reply-To: <1325699559.19044.YahooMailNeo@web160904.mail.bf1.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable FWIW I opened a thread on dev@svn about the time it takes=0Afor svn to comm= it and merge large changesets.=A0 If there's=0Anothing more they can do to = improve the situation, I have=0Aa hunch that migrating the cms to a machine= with SSD's running=0Al2arc cache for zfs will help a lot.=0A=0AWe have a m= achine lying around that would suit that purpose,=0Ajust haven't had the ti= me to bring it up and migrate over to it.=0AEventually tho I'll get the fre= e tuits to make it happen.=0A=0A=0A=0A----- Original Message -----=0A> From= : Joe Schaefer =0A> To: "ooo-dev@incubator.apache.o= rg" =0A> Cc: =0A> Sent: Wednesday, January 4,= 2012 12:52 PM=0A> Subject: Re: suggested CMS workflows for ooo-site =0A> = =0A> ----- Original Message -----=0A> =0A>> From: Dave Fisher =0A>> To: ooo-dev@incubator.apache.org=0A>> Cc: =0A>> Sent: W= ednesday, January 4, 2012 12:40 PM=0A>> Subject: Re: suggested CMS workflo= ws for ooo-site =0A>> =0A>> =0A>> On Jan 4, 2012, at 9:29 AM, Joe Schaefer= wrote:=0A>> =0A>>> =A0 Also Dave get in the habit of checking buildbot for= the=0A>>> =A0 build status of sledgehammer commits instead of waiting=0A>>= > =A0 for svnmailer to figure out what to do with the massive=0A>>> =A0 dif= f it's trying to make sense of.=A0 The url is =0A>>> =0A>>> =0A>>> =A0 http= ://ci.apache.org/builders/ooo-site-site-staging=0A>> =0A>> I do do that, b= ut tend to wait for the email anyway. If there is no reason =0A> to =0A>> = wait that will save time.=0A> =0A> Buildbot performs the commit back as the= final step in the build,=0A> so if buildbot thinks the build has completed= successfully, you=0A> do not need to wait for svnmailer to send out a noti= ce to that effect.=0A> =0A> My experience is that the turnaround between sl= edgehammer commits=0A> and eventual publication is about 1 hour: ~20 min fo= r each step=0A> along the way, all because of svn committing or merging hug= e volumes=0A> of data.=0A> =0A>> =0A>>> =0A>>> =A0 You may want to turn thi= s thread into some basic cms=0A>>> =A0 usage documentation on the incubator= /openofficeorg site.=0A>> =0A>> Yes. I think it is time to rework what is = on the incubator site.=0A>> =0A>> See incubator/openofficeorg/website-loca= l.mdtext.=0A>> =0A>> But not today. I'll let you know when and we can revi= ew it.=0A> =0A> Sounds like a plan.=0A> =0A>> =0A>> Regards,=0A>> Dave=0A= >> =0A>> =0A>>> =0A>>> =0A>>> =0A>>> =A0 ----- Original Message -----=0A>>>= > =A0 From: Joe Schaefer =0A>>>> =A0 To: "ooo-dev@i= ncubator.apache.org" =0A>> =0A>>>> =A0 Cc: = =0A>>>> =A0 Sent: Wednesday, January 4, 2012 11:51 AM=0A>>>> =A0 Subject: R= e: suggested CMS workflows for ooo-site =0A>>>> =0A>>>> =A0 You can always = mimic exactly what the staging site=0A>>>> =A0 does with its builds, namely= target the build to=0A>>>> =A0 a checkout of=0A>>>> =0A>>>> =A0 https://sv= n.apache.org/repos/infra/websites/staging/ooo-site/trunk=0A>>>> =0A>>>> =A0= When the build finishes you can just svn diff that=0A>>>> =A0 tree to see = what you actually changed, but *please*=0A>>>> =A0 do not commit back those= changes yourself.=A0 You=0A>>>> =A0 can nuke all your changes to that tree= with svn revert -R=0A>>>> =A0 so it's handy for testing different types of= sledgehammers.=0A>>>> =0A>>>> =0A>>>> =A0 HTH=0A>>>> =0A>>>>> =A0 ________= ________________________=0A>>>>> =A0 From: Dave Fisher =0A>>>>> =A0 To: ooo-dev@incubator.apache.org =0A>>>>> =A0 Sent: Wednesd= ay, January 4, 2012 11:46 AM=0A>>>>> =A0 Subject: Re: suggested CMS workflo= ws for ooo-site =0A>>>>> =0A>>>>> =0A>>>>> =A0 On Jan 4, 2012, at 8:28 AM, = Joe Schaefer wrote:=0A>>>>> =0A>>>>>> =A0 Given that the size of ooo-site i= s around 9GB, there=0A>>>>>> =A0 are some unique challenges here in dealing= with the CMS.=0A>>>>>> =A0 For the most part tho, the typical workflow of = editing=0A>>>>>> =A0 a few pages on the site, committing them, and publishi= ng=0A>>>>>> =A0 them can all be done reasonably effectively using the CMS= =0A>>>>>> =A0 website.=0A>>>>>> =0A>>>>>> =A0 OTOH, people who need to monk= ey with templates/** or =0A> lib/**=0A>>>>>> =A0 files will trigger full si= te builds and their changes may=0A>>>>>> =A0 materially impact every file o= n the site.=A0 While I've =0A> now=0A>>>>>> =A0 reduced the build time to a= round 4 minutes, the bottleneck=0A>>>>>> =A0 now remains squarely in the ti= me it takes svn to commit =0A> back=0A>>>>>> =A0 those changes and to deal = with merging those changes =0A> during=0A>>>>>> =A0 publication requests.= =0A>>>>> =0A>>>>> =A0 Thanks for your improvements. =0A>>>>> =0A>>>>>> =0A>= >>>>> =A0 In those circumstances I strongly advise you to use the=0A>>>>>> = =A0 publish.pl script on people.apache.org to review and if=0A>>>>>> =A0 ok= publish your changes.=A0 This will eliminate the chances=0A>>>>>> =A0 that= your browser times out a direct publish request to =0A> the=0A>>>>>> =A0 C= MS site, which is a real hassle given that it takes ~15=0A>>>>>> =A0 minute= s for a largeish publish request to be processed.=0A>>>>> =0A>>>>> =A0 I al= ways use publish.pl when I use my sledgehammer ;-)=0A>>>>> =0A>>>>> =A0 I u= sually test my changes with local build_site.pl or =0A>> build_file.pl.=0A= >>>>> =0A>>>>> =A0 My observation is that the biggest bottleneck is more in= the =0A>> creation of the =0A>>>> =A0 email reports. Particularly after p= ublish.pl returns.=0A>>>>> =0A>>>>>> =0A>>>>>> =A0 In the near future we wi= ll be upgrading svn to 1.7 on the =0A> CMS=0A>>>>>> =A0 server which will b= ring in better performance along with=0A>>>>>> =A0 full support for deletio= ns via svn, but I don't expect =0A> the=0A>>>>>> =A0 performance changes to= significantly alter the workflow =0A> I'm=0A>>>>>> =A0 recommending here.= =0A>>>>>> =0A>>>>>> =A0 And please for the sake of others who want to work = on =0A> minor=0A>>>>>> =A0 changes to the site, don't make a sledgehammer t= ype =0A> commit=0A>>>>>> =A0 without following up with an eventual publish = request, =0A> because=0A>>>>>> =A0 publish requests are an all-or-nothing t= ype deal.=A0 That =0A> means=0A>>>>>> =A0 a sledgehammer commit will cause = unreasonable delays for =0A> people=0A>>>>>> =A0 who are trying to publish = minor changes to the site, until=0A>>>>>> =A0 the person who did the sledge= hammer commit follows thru =0A> and=0A>>>>>> =A0 publishes everything.=0A>>= >>> =0A>>>>> =A0 I would recommend that larger template and skeleton change= s =0A> with =0A>> the whole =0A>>>> =A0 ooo-site are done locally and full= y tested before committing to =0A> svn..=0A>>>>> =0A>>>>> =A0 Do you have a= ny recommendations for comparing a locally built =0A> site =0A>> with =0A>= >>> =A0 current production in order to understand how big a sledgehammer = =0A> is =0A>> being built?=0A>>>>> =0A>>>>> =A0 Regards,=0A>>>>> =A0 Dave= =0A>>>>> =0A>>>>>> =0A>>>>>> =0A>>>>>> =A0 HTH=0A>>>>> =0A>>>>> =0A>>>>> = =0A>>>>> =0A>>>> =0A>> =0A>