maven-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kristian Rosenvold <kristian.rosenv...@gmail.com>
Subject Re: Parallel vs weave
Date Fri, 11 Jun 2010 17:40:07 GMT
Interesting info, and good numbers! Weave is a double edged sword;
while it may be promising even
faster builds, by design it has this irritating tendency to run mojos
(of the same execution in different modules)
*exactly* in parallel, more or less simulating how I'd write a unit
test to test thread safety. So weave
has proved to find almost *all* of the concurrency bugs I've located
(where parallel doesn't have any problems).
There are also a couple of minor details regarding the actual
execution planning in weave mode that are still
somewhat open - it's a much more complex execution model than
parallel; but I'm concentrating on
parallel and the ecosystem in general ATM and just making sure weave
works for every project I test.

Antrun itself is threadsafe, but whatever is being executed there is
another matter. If this is committed to cxf
trunk I can/will look at it once I get through the known issues.

Kristian


On Fri, Jun 11, 2010 at 6:57 PM, Daniel Kulp <dkulp@apache.org> wrote:
>
> Well, weave mode doesn't work for CXF.   Ends up with a NullPointerException
> down in antrun-plugin someplace.   Thus, I cannot help you there.
>
> For CXF on my i7 820QM (4 cores, 8 threads) turning off checkstyle/pmd and
> having everything already code generated/compiled (so mostly just running
> tests)
>
> Linear: 26 minutes
> Parallel: 9 minutes  (-T 12 is what I'm using)
>
> Dan
>
>
> On Friday 11 June 2010 12:24:13 pm Kristian Rosenvold wrote:
>> I am picking up some positive tweets here and there, but I'd be really
>> happy if those of you who test parallel building could report back with
>> some numbers in response to this message. I am particularly interested
>> in the difference between parallel and weave (and linear too), mostly to
>> assess that weave is worth the effort.
>>
>> So after tweaking around with threads a bit to find out what runs
>> fastest I'd really enjoy reports like this one (real data):
>>
>> Linear: 50 seconds
>> Parallel: 40 seconds
>> Weave: 29 seconds
>> For a 10 module project using -T 1C on a 6 core box.
>>
>>
>> For those wondering:
>> * Both parallel and weave favorize projects with lots of modules
>>   and "wide" dependency graphs.
>> * Weave is /probably/ quite a lot faster than parallel for projects
>>   with lots of tests fairly evenly distributed among the modules.
>> * Putting all your tests in one module is a bad idea in this context.
>> * Some mojos (like war/ear) can "dwarf" all the others,
>>   effectively limiting the overall potential somewhat.
>>
>> Kristian
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>> For additional commands, e-mail: dev-help@maven.apache.org
>
> --
> Daniel Kulp
> dkulp@apache.org
> http://dankulp.com/blog
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Mime
View raw message