harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark Hindess" <mark.hind...@googlemail.com>
Subject [classlib][build] Observations about build times
Date Tue, 18 Aug 2009 20:12:28 GMT

Fact: If we build all our java code twice, then a fresh build is twice
as fast and a trivial (nothing to do) build is three times faster.
(And the clean target can run 5 times faster.)

While working on cleaning up the clean target, I observed that the
"clean" in the accessibility module took much more time that the
"clean" in the x-net module.  This is a bit odd since the latter is
approximately three times larger so you'd expect if anything it would
be the other way around.

It turns out the "clever" fileset[0] used to find which of the classes
in the "big" top-level compile destination directory belong to which
module, which looks something like:

    <fileset id="classes" dir="../../build/classes">
        <or>
            <!-- only common code case -->
            <present targetdir="src/main/java" />
            <present targetdir="src/main/java">
                <mapper type="regexp"
                        from="^(.*?)(\$$[^/\\\.]*)?\.class$$"
                        to="\1.java"/>
            </present>
        </or>
    </fileset>

Unfortunately, while the patternset concisely expresses our
requirement to only taking class files for which a corresponding
source file exists this is hugely inefficient since it traverses the
very large number of files in the common ../../build/classes tree to
clean (or jar) the relatively small number of classes belonging to a
particular module.  So the reasons the clean in accessibility takes
longer is that it has to check all the classes from every module where
as x-net only has to check those that are left (i.e. those from
x-net).

Patching the build to compile every class twice - once during the big
top-level compile - and again in the module to a different location so
the classes are trivial to select (using the top-level compile on the
classpath) eliminates the need for the "clever" fileset.  Creating the
jars happens much faster.  For clean, modules delete the destdir for
their javac and the top-level just deletes the one from the top-level
compile.  The requirement to piggyback a sanity check the patternsets
during clean goes away because the patternsets are gone.

I thought this was quite interesting/amusing so I thought I'd share
it.  However, I'm really not suggesting that this is the correct
solution but I'm planning to try a few things with a view to improving
things in this area[1].

Regards,
 Mark.

[0] No prizes for guessing which idiot suggested these "clever"
filesets.  They were introduced because they were less error prone
than the manually updated patternset.txt files.  I recall the sound
module was accidentally not updated when new classes were added. See:

  http://markmail.org/thread/rjqrgudag5jxw3wt

[1] I think Alexey Petrenko might have had the right idea when he
suggested keeping the simple patternset files but automatically
generating them to avoid trivial mistakes.



Mime
View raw message