openoffice-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kay Schenk <ksch...@apache.org>
Subject Re: Buildbots update
Date Sun, 24 Jul 2016 15:55:03 GMT
On Fri, Jul 22, 2016 at 12:09 AM, Damjan Jovanovic <damjan@apache.org>
wrote:

> I've progressed much further, and openoffice-fbsd-nightly,
> openoffice-linux32-nightly, openoffice-linux64-nightly, and
> openoffice-linux64-rat are now building, while openoffice-linux32-snapshot
> is only temporarily breaking due to SourceForge issues. I've also made some
> interesting discoveries about the Windows buildbots.
>

​Lookin' pretty good so far! A BIG thank you again for all your work on
this!
Dealing with the buildbot configurations is quite a challenge!
​


>
> I had to revert r1736692 (in r1753709), setting main/extensions.lst back to
> generic https:// SourceForge URLs, as the mirror-specific http:// URLs are
> now broken, which was causing all buildbots that use
> --enable-bundled-dictionaries to fail. Enough buildbots support https now
> to make this a net benefit.
>
> Also had to upload openssl-0.9.8zg to ooo-extras for the
> openoffice-linux32-snapshot, but the most recent build failed because it
> couldn't download some dictionaries/languages from SourceForge, which I am
> generally finding to be too flaky.
>
> I think we should either run ./bootstrap multiple times on the buildbots,
> or list SourceForge URLs several times in external_deps.lst and
> extensions.lst, to compensate for SourceForge's unreliability. Buildbots
> should immediately fail the build if ./bootstrap fails, as there is no
> hope: ./bootstrap only downloads dependencies needed for the build, and if
> any are missing, the build will definitely fail, and burning CPU for up to
> 5 hours is pure waste. Alternatively we should find a more reliable
> ooo-extras hosting solution than SourceForge. We could also cache
> dependencies offline between builds, but I am guessing that has licensing
> issues?
>
> That leaves the Windows bots.
> aoo-w7snap is still missing LWP::Protocol::https.
> aoo-win7 was failing to delete the old build files (rm: cannot remove
> `build/ext_libraries/apr/wntmsci12.pro/misc/build/apr-1.4.5/Makefile.win':
> Device or resource busy).
> Something seems to be keeping that file open even after the previous builds
> are over.
> According to ext_libraries/apr/makefile.mk, the BUILD_ACTION on Windows
> is:
> INCLUDE="$(INCLUDE);./include"  nmake -f Makefile.win buildall
> so I suspected nmake hangs.
>
> I patched the build script to run "ps -W" (results at
> https://ci.apache.org/builders/aoo-win7/builds/348/steps/ps/logs/stdio)
> which showed 4 nmake processes, 2 from March 30, 2 from April 4, part of
> orphaned trees also containing perl, sh, and dmake.
> Killing nmake (through hacks to the buildbot script, as I still don't have
> remote access) had no effect - deleting apr-1.4.5/Makefile.win was still
> giving the same error.
> Even killing dmake, sh and perl (which had to be done in creative ways on
> that dodgy OS - some through taskkill, some through Cygwin's kill) still
> had no effect.
> After all Cygwin processes were dead, that error was still coming up!
>
> So I started looking at non-Cygwin processes. DEVENV.EXE and DEVENV.COM
> had
> the same March 30 / April 4 start times as the hung process trees, and
> killing them *finally* allowed apr-1.4.5/Makefile.win to be deleted.
>
> I'll experiment a lot more over the weekend, but right now I think the
> problem could be that the buildbot runs VSVARS.BAT to set up the Visual
> Studio environment, which (presumably) contains the evil DEVENV that break
> the build and locks files while hung. On my own Windows VM I don't run
> VSVARS.BAT, and I can't reproduce the problem. Do the rest of you that
> build on Windows use it?
>
> That buildbot is currently running out of disk space, and it doesn't help
> that we "svn co" and then "svn export" a second copy. Originally the
> buildbot script used other tricks, like "svn switch", or keeping an SVN
> checkout across builds that was just updated and then exported from for
> each build, but some time ago I switched to a full "svn co" because it was
> too unreliable (eg. files can get locked and need "svn cleanup"). With a
> full checkout there is no need to export, as we get a fresh copy each time.
> I'll overhaul that buildbot script and try make it simpler and cleaner.
>
> On Tue, Jul 19, 2016 at 8:17 PM, Damjan Jovanovic <damjan@apache.org>
> wrote:
>
> > Hi
> >
> > I contacted Infra on HipChat and asked them to fix the buildbots I could
> > find with the Perl LWP::Protocol::https problem (aoo-w7snap,
> > openoffice-fbsd-nightly, and openoffice-linux32-nightly) or give me
> access
> > to do it myself, and @pono fixed at least the openoffice-linux32-nightly
> > bot.
> >
> > The other buildbots are either failing earlier or failing for other
> > reasons. For example openoffice-linux64-nightly was failing to download
> > openssl ("500 Can't connect to www.openssl.org:443"), but I've uploaded
> > it to ooo-extras and it's gotten further in the build I am forcing now.
> >
> > Damjan
> >
> >
>



-- 
--------------------------------------------------------------------------------------------------------
Kay Schenk@Apache OpenOffice

“Things work out best for those who make the best of how things work out.”

-- John Wooden

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message