lucy-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marvin Humphrey <>
Subject Re: [lucy-dev] canonical build systems
Date Sat, 28 May 2011 18:11:59 GMT
On Sat, May 28, 2011 at 05:20:34AM -0700, Joe Schaefer wrote:
> Looking over the codebase with an eye towards future language bindings, I
> really think we'll be in for some serious maintenance headaches if we stick
> to the plan to have a fully independent build system for each lang.

Much of the build logic which has accumulated in
trunk/perl/buildlib/Lucy/ is there for reasons of expediency rather
than design.  Factoring out Lucy::Build into shared chunks will be a natural
and important task as we add new language bindings.

Logically, each host language *will* have its own build system -- it would be
silly to forswear Perl when building Lucy's Perl bindings and attempt to
compile and install an XS module using bare Make, Rake, or Python distutils.

So the goal here has to be *minimizing* what we do for each host language.
That, I'd be excited about, and it is absolutely feasible.

> My experience elsewhere tells me that finding humans willing to maintain
> build system changes as the codebase evolves will be a challenge.

If we were to port Lucy::Build as is to other languages, I agree that we would
be setting ourselves up for bitrot -- especially for bindings to less
ubiquitous languages like Lua or Go.  We need to bring down the cost for each

> Rather, I would like to see charmonizer and clownfish get one build system
> that works for all langs.

Let's consider Charmonizer first, since that's the easiest layer to tackle in
isolation, and since several Lucy committers have hacked on it enough to
become familiar with it.

I think we should consider changing Charmonizer's workflow to resemble that of
Metaconfig and Autoconf more closely, and have it generate a single monolithic

    Metaconfig:  Configure
    Autoconf:    configure
    Charmonizer: configure.c

Elaborating on the analogy, the incantation for building a Charmonizer-enabled
C project would theoretically look like this[1]...

    make conf      <--- replaces ./Configure or ./configure
    make test
    make install

... although in most cases the "conf" target would be a dependency of "all"
and so the canonical incantation would probably look like this:

    make test
    make install

Note that since configure.c would be cross-platform-compatible C code rather
than Bourne shell, the same sequence of build commands can be used
*everywhere*. :)

This "configure.c" file would be a concatenation of everything that's
currently broken up into individual C files under trunk/charmonizer/src --
excluding test files and modulo renaming some static variables.

Once we're down to a single C file, building it becomes trivial.  We can zap
most or all of ACTION_charmonizer from Lucy::Build, replacing it with e.g.
simple Makefiles or one-line compilation commands.

The naive way to get this done would be to create one Perl/Python/Ruby module
per current Charmonizer module, each with the current C code embedded.  These
modules would then combine their output to produce "configure.c".

(Eventually, Charmonizer might expose a public Perl/Python/Ruby API allowing
programmers to select configuration probes and tests a la carte -- but that's
down the road.)

> If that means picking perl/python/whatever to do the job so be it.

In think the guiding principle ought to be that it's OK if we impose
significant prerequisites on Lucy's developers, but not on our users.

With that in mind... For serving as Charmonizer's foundation, Python, Perl and
Ruby would all be roughly the same from a practical standpoint, fanboys be
damned.  I'd lean towards Python 3, because Python's liberal licensing
traditions are most in tune with Apache's.  

Under this proposal, building Lucy from a Subversion checkout would require
Python 3, but building it from a CPAN tarball would not.

> If that means creating a Windows-friendly DSO that's doable
> too (I can help here for sure).

Such an approach implies not just a canonical build system, but a single
shared object to be used by multiple host language bindings.  At present,
that's not feasible because each Lucy shared object contains compiled code
which is specific to one major version of one host language.  Changing that
would be a major undertaking.

> OTOH if we do continue down the path of fully independent
> per-lang build systems, we absolutely must track
> build changes on a per-lang basis with buildbot or jenkins.


We were always going to need this.

Marvin Humphrey

[1] This is a variant on Nate's 'make config; make all; make install' proposal
    from last November: <>.

View raw message