ant-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David Weintraub" <qazw...@gmail.com>
Subject Re: Applying ParallelTask
Date Tue, 06 Nov 2007 13:15:30 GMT
You may have to do a bit or rewriting in order to use the <parallel>
task. This isn't unusual. Most software has to be rewritten in order
to take advantage of multiple processors, so this isn't unusual.

First question: Are Module "B" and "C" independent of each other? That
is, can you create a release that is just Module "B" or Module "C"? If
not, then they should share the same build.xml file. That way, you can
let your build.xml file determine the build order of your entire
project. I've seen too many build.xml that rely on <antcall> tasks to
be executed in specific order. This makes everything harder to
maintain.

If Module B and C are really independent upon each other and let's say
that Module A is a foundation class for your project, then Module B
and C's build.xml files should contain the dependency information
inside of them. That is if Module B is dependent upon Module A, it
should contain the dependency to all Module A's build.xml and not in
your master build.xml file. This information shouldn't be in your
master build.xml file.

Dependency checking in Ant is done by comparing the timestamps of the
source files against the timestamps of the target's output. You
shouldn't do anything that breaks this such as using the <move> or
<delete> tasks. If you are having problems with build dependency
rebuilding, use the <uptodate> and the AntContrib <outofdate> tasks to
fix the problem. That way, you avoid unnecessary rebuilds and fix
multiple dependency issues when you're using more than one build.xml.

In a perfect world, all you would need is one parallel task that calls
both Module B's and Module C's build.xml file, and Module A's
build.xml will be called if needed. Ant would have some sort of
locking mechanism to prevent both Module B and Module C attempting to
build Module A at the same time. Unfortunately, I doubt that Ant is
this intelligent.

That means that you should have your parallel task have a dependency
on building Module A and calling Module B and Module C in parallel. If
your dependency checking is correct, neither Module B or Module C will
rebuild Module A. One or the other might attempt to run the "Build
Module A" task, but they would already see that Module "A" is built,
and go on their merry way.

This isn't perfect. Otherwise, you shouldn't have to mention any
dependency information in your master build.xml (since this is
contained in each module's own build.xml file). However, normally
Module A is a set of foundation classes that your whole project
structure is dependent upon, so you can safely assume that Module A
has to be built first because most (if not all) or your other modules
will depend upon it.

So, your first answer would be the best in your situation: That is,
have your master build.xml build any foundation classes, then call the
modules that actually make up your project (and may or may not depend
upon your foundation classes) in parallel.

On 11/6/07, Arseny <spanishguitar2005@gmail.com> wrote:
> Hello!
> After moving our Ant server to Intel Core Quadro I decided to make our
> ant build in parallel.
> I studied the ParallelTask in ant, but found the following problem in
> applying it to our project.
> The project consists of a hundred separate modules (jars) and its
> build.xml is
> made in terms of targets. I mean, it is like:
> <target name="rebuild_all" depends "ModuleA, ModuleB, ModuleC"/>
>
> <target name="ModuleA" depends="">
>    <ant dir="ModuleA_dir" antfile="build.xml" target="rebuild"/>
>  </target>
>
> <target name="ModuleB" depends="ModuleA">
>    <ant dir="ModuleB_dir" antfile="build.xml" target="rebuild"/>
>  </target>
>
> <target name="ModuleC" depends="ModuleA">
>    <ant dir="ModuleC_dir" antfile="build.xml" target="rebuild"/>
>  </target>
> ...
>
> I found two ways in making this tasks run in parallel:
> 1.
> <target name="rebuild_all" depends "ModuleA, ModulesParallel"/>
>
> <target name="ModuleA" depends="">
>    <ant dir="ModuleA_dir" antfile="build.xml" target="rebuild"/>
>  </target>
>
> <target name="ModulesParallel" depends="ModuleA">
>     <parallel>
>       <ant dir="ModuleB_dir" antfile="build.xml" target="rebuild"/>
>       <ant dir="ModuleC_dir" antfile="build.xml" target="rebuild"/>
>     </parallel>
>  </target>
>
> The problems with the first way are:
> 1) We lose information on dependencies between certain modules (now I do not
> know which of the modules: ModuleB or ModuleC (or both) depends on ModuleA)
> 2) Difficult to separate in parallel parts all of the numerous tasks
>
> 2.
> <target name="rebuild_all" depends "Modules"/>
>
>  <target name="Modules" depends="">
>     <parallel>
>       <antcall target="ModuleA"/>
>       <antcall target="ModuleB"/>
>       <antcall target="ModuleC"/>
>    </targe>
>
> <target name="ModuleA" depends="">
>    <ant dir="ModuleA_dir" antfile="build.xml" target="rebuild"/>
>  </target>
>
> <target name="ModuleB" depends="ModuleA">
>    <ant dir="ModuleB_dir" antfile="build.xml" target="rebuild"/>
>  </target>
>
> <target name="ModuleC" depends="">
>    <ant dir="ModuleC_dir" antfile="build.xml" target="rebuild"/>
>  </target>
>
> The problem with the second variant is that it simply does not work:(
> As I see, it has racing conditions when several threads start to rebuild
> the same task, that
> results in losing ability to make directories, rebuild jars...
>
> If you have any ideas on how to solve this problem, any help would be
> appreciated!
> Thanks for reading my message!
>
> Arseniy.
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@ant.apache.org
> For additional commands, e-mail: user-help@ant.apache.org
>
>


-- 
--
David Weintraub
qazwart@gmail.com

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@ant.apache.org
For additional commands, e-mail: user-help@ant.apache.org


Mime
View raw message