ant-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chuck Holzwarth <chuck_holzwa...@yahoo.com>
Subject Re: AW: extended parallelism
Date Wed, 19 Dec 2007 16:15:35 GMT
In the formula, the precedence order is the non parallel execution takes precedence over the
parallel execution. It will be necessary to document this information. 

The usage of math notation in the dependency list would seem to be a break from the ant standard
of using tags for everything. Perhaps this is something for the apache contrib library. With
a change in standard ant to ignore the "( and ")" and treat the "|" the same as the ",". This
way overriding would only occur if the contrib library is loaded and sequential functioning
will continue if it is not. Will the use of "|" cause problems on Unix type systems? It may
be better to choose a different set of symbols since "(", ")" and "|" have specific meanings
already. 

I can see why a tag based notation is not going to work. For example a,b,c,d,e,f  where b,c
and d,e are each parallelizable as a set but b,c,d,e are not. This is easily represented by
the equation a, (b | c), (d | e), f.

I, for one, would welcome the option to complete some parts of the build in parallel.

Also, will the parallel operation be the standard behavior or the non parallel? I would vote
for the parallel option to be the standard since most systems now come with multiple cores.

Robin Chaddock <robin.chaddock@iplay.com> wrote: The target "depends" attribute already
gives all the information you need to be able to determine which targets can be executed in
parallel?

As I mentioned earlier the only road-block at the moment to a totally transparent mechanism
for supporting parallel execution is the dubious implicit dependencies that the documentation
for the "depends" attribute makes reference to. (and the great many build scripts in the wild
that rely upon it :( )


  ----- Original Message ----- 
  From: Chuck Holzwarth 
  To: Ant Users List 
  Sent: Wednesday, December 19, 2007 2:33 PM
  Subject: Re: AW: extended parallelism


  The problem with giving a command line option is that ant is effectively a set of rules.
The rules form a tree and the dependencies give a list of targets that are to be executed
before the current target. There would have to be some intelligence in the build file that
would show ant which tartets could be completed in parallel. For example, if you are using
ant to compile C programs, the obj files that the executable requires must be completed before
the linkage stage. There are areas where parallel processes can be leveraged, but ant will
not be able to determine those places. In my experience, even the programmers who have written
the code have difficulty identifying those places.

  In my case, there are times where parallem processing could be helpful. I retrieve the latest
copy of all files from the repository for builds, I retrieve the CM log information, I create
the release boiler plate file using information in a property file that is loaded at startup.
These could all be executed in parallel. One part of the build that could not be executed
in parallel is creation of the zip file that includes only the HTML files that changed since
the previous release. These files are copied from the complete release zip files and can't
be attempted until the zip files with the complete set of files is available.

  Jan.Materne@rzf.fin-nrw.de wrote: > It may be better to stay away from a grammar that
gives the 
  > rules and use something like:
  > 
  >   
  >  

  > 
   ... 
  >  
  >  


  You could provide another executor ;)
  In my approach I dont want the user to change the buildfile. Just set
  the property on command line.
     ant -Dant.executor.class=ParallelExecutor ...

  > How do you propose to handle potential fatal/non fatal 
  > errors? 

  The implementation collects thrown BuildExceptions from the target.
  A target will be started, if no dependend target has failed. 
  Why not try to build the jar only because the test fails? If there is a
  dependency you should 
  describe that dependency using the depends-attribute.



  >If target a exits due to an error, should there be an 
  > option to kill a or allow it to complete (similar to 
  > failonerror="yes/no")? If both a and (b,c) must succeed for 
  > d, should a be killed if b or c fails? 

  If you want to have that, use 
    
  If there is no dependency - why stop the build?


  > It was not difficult, but error handling was always the issue.

  My biggest problem is spreading all the logs. In a single thread you
  have output like
    target-1
    target-1
    target-2
    target-2
  When parallelizing that you could earn
    target-2
    target-1
    target-2
    target-1

  I thought about using a logger per thread
      Logger log = Logger.getLogger( targetName + "." + threadNumber );


  Jan

  ---------------------------------------------------------------------
  To unsubscribe, e-mail: user-unsubscribe@ant.apache.org
  For additional commands, e-mail: user-help@ant.apache.org




  Thank you,
  Chuck Holzwarth
  (804) 403-3478 (home)
  (540) 335-3171 (cell)
         
  ---------------------------------
  Be a better friend, newshound, and know-it-all with Yahoo! Mobile.  Try it now.

________________________________________________________________________
E-mail is an informal method of communication and may be subject to data corruption, interception
and unauthorised amendment for which I-play, a trading name of Digital Bridges Ltd will accept
no liability. Therefore, it will normally be inappropriate to rely on information contained
on e-mail without obtaining written confirmation.

This e-mail may contain confidential and/or privileged information. If you are not the intended
recipient (or have received this e-mail in error) please notify the sender immediately and
destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material
in this e-mail is strictly forbidden.

(C) 2005. I-play is a trademark and trading name of Digital Bridges Limited. All Rights Reserved.
________________________________________________________________________
This message has been checked for all known viruses by the 
MessageLabs Virus Scanning Service. For further information visit
http://www.messagelabs.com/stats.asp


Thank you,
Chuck Holzwarth
(804) 403-3478 (home)
(540) 335-3171 (cell)
       
---------------------------------
Never miss a thing.   Make Yahoo your homepage.
Mime
  • Unnamed multipart/alternative (inline, 8-Bit, 0 bytes)
View raw message