harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Salikh Zakirov <salikh.zaki...@gmail.com>
Subject [drlvm][PATCH/RFC] GNU Make-based build system for DRLVM
Date Wed, 01 Aug 2007 04:28:40 GMT

Recently I've been trying out some massive code reorganizations in DRLVM
code in order to improve its modularity (especially concerning
interpreter vs. JIT issue), and that involved mass moving of source
files between components and frequent recompilations, as I am trying
to use resulting compiler and linker errors as a guidance on what else
needs to be done.

Unfortunately, this kind of exercise proves the current ant-based build
system highly inconvenient, as it has following problems:

* It neither cleans up stale object files, nor uses object file lists,
  so in case .cpp file was moved, the subsequent rebuild will link the
  stale object file. 

  The obvious workaround is to do a 'build.sh clean', but this is very
  time consuming (~5 min on my machine)

* Even in case of simple changes to several .cpp files, the compilation
  process takes unacceptably long time. In the extreme, the 'build.sh'
  command with no changes at all takes about 51 seconds on my machine.

* Ant does not support parallel builds (and my machine is dual-core)

In order to make the workflow more effcient (so as not to spend too much
time on recompilation), I've created a build system using GNU Make from
scratch. It uses a little bit of makefile generation on-the-fly, but
it is relatively simple: 
(a) dependencies are tracked using 'makedepend' 
(b) individual file compilation rules are generated from source lists by
a very simple shell script, which generates output like the following:

    $ gen-sources-make *.cpp
    OBJECTS :=
    DEPENDS :=

    OBJECTS := $(OBJ)/a.o
    $(OBJ)/a.o: a.c
	    makedepend $(CPPFLAGS) $< > a.d
	    $(CC) -c -o $@ $<

    ... (repeat the above for each source file)

The end result is included into the makefile. This is needed for two
reasons: (1) flatten object file directory (i.e. source files from a
deep tree will have a flat list of corresponding object files)
(2) to get the list of object files explicitly, in order to prevent
erroneous linking of stale object files.

The resulting build system has the following advantages over current
ant-based system:

+ No-recompilation run is fast: 3 sec (with hot cache) vs. 51 sec with
  current system
+ Stale object files are not linked, even without cleaning
+ Parallel make is supported
+ The output is nice and clean, without noise
+ The configuration files (e.g. vm/em/Makefile) are shorter, and the whole
  build process is far easier to understand (to me anyway)

However, it also has some disadvantages, compared to the ant system
* It only covers building of the native targets (libraries and
  executables), so testing and class/jar building is not covered.
  If this system is accepted, it will probably be called from ant.
* It does not support multiple file compilation with a single compiler
  invocation, and executes compiler once for each source file, which
  may be a real performance problem for some platforms
  (e.g. Windows/Intel compiler). While some hacks to alleviate this
  issue exist for initial compilation, I am not aware of any general
  Make-based solution.

The result is an attached patch. I do not file a JIRA with patch because
it is obviously not (yet) suitable for inclusion. 
I would like to hear the comments on the approach and general view on
whether DRLVM needs yet another one build system.

 build/Makefile               |   58 +++++++++
 build/gen-sources-make       |   55 +++++++++
 build/makerules.inc          |  263 ++++++++++++++++++++++++++++++++++++++++++
 vm/em/Makefile               |   25 ++++
 vm/gc_cc/Makefile            |   21 ++++
 vm/gc_gen/Makefile           |   19 +++
 vm/interpreter/Makefile      |   26 ++++
 vm/jitrino/Makefile          |   47 ++++++++
 vm/port/Makefile             |   28 +++++
 vm/port/src/encoder/Makefile |   11 ++
 vm/thread/Makefile           |   30 +++++
 vm/thread/jthread/Makefile   |   21 ++++
 vm/vmcore/Makefile           |   62 ++++++++++
 vm/vmi/Makefile              |   22 ++++
 14 files changed, 688 insertions(+), 0 deletions(-)

In addition to disadvantages mentioned above, it is not complete, and
misses following items, though these can be implemented easily
provided there is enough interest (as the system already does everything
I personally need). 

* Configuration is currently written for Linux/ia32/gcc/debug only.

* Deployment of built targets is currently rather crude (using cp -u)

* I've just noticed that some changes happened after the I started doing this,
  so the Makefiles with source lists will need some update too.

Comments are welcome.

View raw message