openoffice-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcus <marcus.m...@wtnet.de>
Subject Re: Thoughts on a new build system for AOO
Date Fri, 05 Feb 2016 18:54:32 GMT
Am 02/05/2016 07:32 PM, schrieb Damjan Jovanovic:
> Hi
>
> With the recent buildbot saga and Patricia's Windows building nightmare, a
> number of serious issues were highlighted which got me thinking about how
> our build system could be improved.
>
> The problems with building AOO were researched and documented years ago (
> https://wiki.openoffice.org/wiki/Build_Environment_Effort) and a very good
> analysis was done in https://wiki.openoffice.org/wiki/Build_System_Analysis
> which is worth a read.
>
> Their solution was gbuild: a build system using GNU make's new "eval"
> feature to rewrite the makefile from within, which would generate one giant
> dependency graph of every file in AOO and produce reliable, stable,
> deterministic, and maximally parallelized builds, with minimal rebuilding
> on changes. The problem is, the full benefits of gbuild can only be seen
> once every module is converted to gbuild, and with only about 10% having
> been converted so far, it doesn't even try to build a complete project-wide
> dependency graph, but rather gets invoked per-module just like dmake.
> Converting dmake modules to gbuild is very difficult: so far I only managed
> to convert main/formula, but main/animations was mysteriously segfaulting
> AOO...
>
> Besides, LO has completed converting every module to gbuild, guinea pigging
> gbuild perfectly for us ;-). How has it gone for them? One of our
> committers, jani, evaluated LO's gbuild in April 2013 in a thread here
> called "Make x Dmake x Build", where he said how it's "not particulary fast
> either", and called mkdir 431 times on a directory that already existed.
> And worst of all, gbuild uses its own language with horrible syntax (eg.
> LISP-like syntax, spaces around commas not allowed, blank lines sometimes
> mandatory and at other times forbidden), is almost completely undocumented,
> and very difficult to debug - a maintenance nightmare.
>
> So what other alternatives are there?
>
> There was a comment in
> https://wiki.openoffice.org/wiki/Build_Environment_Effort/New_Build_System_Requirements
> that got my attention:
>
> "One might consider using Python as a tool for _all_ non-standard build
> tasks. Python is easy to distribute and install on all platforms. However,
> it is also a very fat dependency and thus it is questionable, if it would
> be worth the effort just to get rid of cygwin (or another Unix environment
> on windows)."
>
> We already build and use Python, and there is a good build system written
> in Python: SCons (http://www.scons.org). It's modern yet well established,
> having started in the year 2000, and is used by a number of open source and
> commercial projects. It's MIT licensed so we can include it as a
> dependency. The build scripts are simple Python scripts and can use the
> Python language and libraries, though you don't need to know (much) Python
> to use it. It's well documented. It's debuggable and has options to print
> dependencies and explain actions taken. It supports all the operating
> systems and architectures we support (even unofficially), including Cygwin,
> *BSD, Solaris, OS/2. It's "local" package is designed for easy embedding
> into a pre-existing project so it runs straight after extraction with no
> need to install it system-wide, and is only 445 kB.
>
> It has many features we need or would benefit from:
> * It's able to parse included headers from C/C++ files and use them as
> dependencies.
> * It can generate Visual Studio project files.
> * Supports many languages including C, C++, D, Java, Fortran, Yacc, Lex.
> * Builds zip and tar archives.
> * Just like gbuild, it has a global dependency view and can fully
> parallelize building files across module boundaries while maintaining
> stability.
> * Using Python APIs instead of *nix CLI tools might eventually result in us
> being able to ditch Cygwin entirely, not to mention better performance due
> to less subprocess spawning.
> * Unlike classical make tools which only use mtimes, it determines what to
> rebuild using several configurable options, including MD5 checksums of file
> contents, meaning changes which don't result in different binaries (eg.
> formatting or comment changes) will only recompile changed files and stop
> there, because it knows the object files are the same as before, so no
> linking or downstream rebuilding needs to be performed.
>
> In other words SCons does most of what gbuild is supposed to be good for,
> and more. I see it outperforming LO's pure gbuild, while being more
> readable, maintainable, debuggable, and a sight for sore eyes after the
> triple whammy of dmake, gbuild and build.pl we've endured for so long.
>
> If people are happy, I'd suggest a migration plan where first scons is
> added as a dependency, then the new version of serf is built with it (see
> i126312), then gbuild modules are ported to scons until gbuild is
> eliminated, then dmake modules are ported to scons until dmake is
> eliminated, and finally scons replaces build.pl.
>
> Thoughts?

thanks for your analysis and suggestion.

I'm not a developer in the traditional way. So, I don't know anything 
about build systems and I've to believe your points - especially about 
the dis-/advantages for gbuild and scons.

To migrate the gbuild-using modules first is clever as it would be a 
nightmare to have a growing mix of dmake, gbuild and scons together.

Eliminating the cygwin stuff on Windows could be another big advantage 
for developers on Windows.

Last but not least the famous question about participation. If you would 
volunteer to start/do the migration then it would be great. But in 
parallel it could also become a life time task. ;-P.

But, yes, using scons sounds a good idea and doing a test with some 
modules that are working with gbuild could be a good start.

Marcus

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org


Mime
View raw message