ant-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeffrey E Care <ca...@us.ibm.com>
Subject Re: Parallel processing option for subant?
Date Wed, 12 Oct 2005 12:32:07 GMT
Steve Loughran <stevel@apache.org> wrote on 10/12/2005 07:50:04 AM:

> > Would that matter? All property reads/writes would be confined to the 
> > thread there the project is being executed. The main problem I think 
would 
> > be in the logger, but I've modified our custom logger to do thread 
> > correlation & buffering; these changes should easily map to Ant's 
> > DefaultLogger.
> 
> Input is another issue.

That is certainly true; I had not considered the <input> task, mostly 
because in WAS we don't/can't use it. It would have to be documented in 
the manual that using parallel subant to build projects that use the input 
task would be verboten.

> > If there's a lot of general concern/opposition about this function I 
don't 
> > want to take the time to develop a patch only to see it languish in 
> > bugzilla or be outright rejected (which is why I wanted to discuss it 
here 
> > first). Would it be appropriate to add this function and put a warning 
in 
> > the manual that using the parallel execution could potentially be 
> > dangerous?
> 
> to an extent <parallel> is the most prone to race conditions, as it is 
> running stuff in the same project. a subant running in parallel is 
> likely to share much less, just a FileUtils instance and maybe some 
> references handed down.

Is it same to assume that we'll be running against JDK 1.2 or greater? If 
so we could possibly convert FileUtils (or any other class with a static 
"getInstance" singleton accessor) to use ThreadLocal.

> > The reason why I'm sort of pushing so hard for this function is that 
we've 
> > already implemented something similar in the WAS build tools and it's 
> > shaved 25-30% off of some of our builds with no adverse effects.
> 
> I guess we can view that as a functional test :) What hardware were you 
> running on? How many CPUs?

I can get exact specs if you want, but we have run this in our production 
build lab on AIX, Linux, Solaris, Windows, HPUX and OS400. Most, if not 
all, of these boxes are 2-way or better, and right now we're running 
2-threads per proc; the iSeries (OS400) is a 12-way so it's running 24 
build threads in addition to the main thread. It's also run on zOS, but 
the parallel function does not (currently) get exercised there due to how 
the source code is organized on the mainframe.

-- 
Jeffrey E. Care (carej@us.ibm.com)
WebSphere v7 Release Engineer
WebSphere Build Tooling Lead (Project Mantis)
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message