ant-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jose Alberto Fernandez" <jalbe...@cellectivity.com>
Subject RE: Need a target meta data for parallel executor.
Date Thu, 20 Jan 2005 11:52:22 GMT

I think you misunderstood my syntax so I will try again with diferent 
syntax to make it more clear:

Lets use  "||" as the non-deterministic separator for dependencies.
If you say:
   <target name="foo" depends="a||b"/>

this means "foo" must be executed  after "a" and after "b"
but there is no restriction in the execution order of "a" and "b".
If on the other hand you write:

    <target name="foo" depends="a,b"/>

this means "foo" must be executed after "a"  and after "b"
and "a" must be executed before "b".

Today, ANT encodes the above restrictions when constructing  the
dependency
order before applying a topological sort and executing things in the
order. 
This includes the dependency between
"a" and "b" above. The "||" operator  just means that there is no
dependency
between "a" and "b" in the graph and hence "a" and "b" will have the
same 
position in the topological sort.

Based on this a parallel executor just need to execute all targets at
the same
position in the order in parallel, a sequential executor will just pick
some
arbitrary order for those targets.

Notice that the topological sort is global and hence guarantees the
correct ordering 
of the targets no matter how complicated the dependency graph.

Now, some thoughts about syntax:

I am sure that for reasons of backward compatibility we would not be
able to use
";" or "||" or any other symbol, being there before. But maybe we can
use 
a diferent way to encode the information, for example:

	<target name="foo" dependantsets="sA, sB" sA="a" sB="b"/>

a more complex case would be:

	<target name="bar" dependantsets="s1, s2, s3" s1="a,b,c"
s2="d,e,f" s3="g"/>

This would be a more backward compatible way to say:

	<target name="bar" depends="a,b,c||d,e,f||g"/>

Jose Alberto

> -----Original Message-----
> From: Alexey N. Solofnenko [mailto:A.Solofnenko@mdl.com] 
> Sent: 19 January 2005 19:16
> To: Ant Developers List
> Subject: Re: Need a target meta data for parallel executor.
> 
> 
> Currently the executor uses only dependency information - if target A 
> depends on B, A can be executed only after B is completed. If 
> there is 
> no direct on indirect dependency between two targets - they can be 
> executed in parallel.
> 
> I also wanted to implement dependencies with semicolon 
> inside, but it is 
> more difficult than I thought at first. For example in this case:
> 
> <target name="clean-build" depends="clean;build"/>
> 
> not only "clean" should be executed before build, but also all not 
> common dependencies of "clean" should be executed before any 
> not common 
> dependencies of "build", and, if there are a lot of such 
> declarations, 
> the ordering becomes very complicated. Fortunately it is 
> still possible 
> to write:
> 
> ant clean build
> 
> to execute the targets in some order.
> 
> - Alexey.
> 
> Jose Alberto Fernandez wrote:
> 
> >Alexey,
> >
> >Can you describe a little how this parallel executor works? 
> How do you 
> >specify things that can run in parallel and those that need to run 
> >sequentially due to dependencies.
> >
> >One thing I wanted in along past was to allow specifying in the 
> >dependencies something like:
> >   depends="a,b,c;d,f,g;h"
> >
> >which means a, b,and c must be executed sequentially, so are 
> d, f, and 
> >g; but there is no restriction in the order between "a,b,c", "d,f,g" 
> >and h.
> >
> >Now the regular executor willjustpick an order and do it. But a 
> >parallel one may execute all the three groups in parallel. This will 
> >give you some control
> >on the way parallelism is applied may remove the need for 
> mutex and the 
> >corresponding deadlocks they may introduce.
> >
> >Does the parallel executor ideas arein any way simillar? I really do 
> >not know.
> >
> >Jose Alberto
> >
> >  
> >
> >>-----Original Message-----
> >>From: Alexey N. Solofnenko [mailto:A.Solofnenko@mdl.com]
> >>Sent: 19 January 2005 18:40
> >>To: Ant Developers List
> >>Subject: Re: Need a target meta data for parallel executor.
> >>
> >>
> >>So, would do you propose? Is it something like:
> >>
> >><project>
> >>  <property name="org.apache.ant.target-mutexes"
> >>value="xxx,yyy;aaa,bbb,ccc"/>
> >></project>
> >>
> >>The target mutual exclusion seems static and nobody should be
> >>changing 
> >>it from an outside (only build script developer should know 
> >>about them). 
> >>On the other hand adding even more ANT constructs for every 
> >>small detail 
> >>seems an overkill. If we add a meta-info facility, it should 
> >>cover this 
> >>and any other future uses. And the meta-info will be also 
> ignored by 
> >>code that does not understand it (for example old ANT will be 
> >>still able 
> >>to run build files prepared for parallel execution).
> >>
> >>- Alexey.
> >>
> >>Matt Benson wrote:
> >>
> >>    
> >>
> >>>Alexey:  Since the Executor classname itself is
> >>>specified as an option, rather than in a buildfile, my 
> opinion would 
> >>>be that specifications that interact with an Executor should be 
> >>>externally configurable. At the same time it would be nice to 
> >>>maintain Target exclusivity information with the targets 
> themselves.
> >>>My first instinct would be to use project-level
> >>>properties.  You should want to execute the default ""
> >>>target first in any event, I would think, so at that
> >>>point project-level properties would be available, yet
> >>>can still be overridden externally.  I would expect
> >>>these properties to be of a dynamic nature, but you
> >>>should be able to use PropertySets to help you with
> >>>that.
> >>>
> >>>HTH,
> >>>Matt
> >>>
> >>>
> >>>--- "Alexey N. Solofnenko" <A.Solofnenko@mdl.com>
> >>>wrote:
> >>>
> >>> 
> >>>
> >>>      
> >>>
> >>>>Hello,
> >>>>
> >>>> I am experimenting with parallel executor.  One of
> >>>>the things required
> >>>>for it is to be able to prevent some targets (mostly
> >>>>test targets) to be
> >>>>executed simultaneously. It is possible to add
> >>>>something like 
> >>>>"mutex-names" attribute to Target, but if we do
> >>>>that, there seems no end 
> >>>>of adding many more attributes that make sense only
> >>>>in some specific 
> >>>>situations. Instead of that we could use XML
> >>>>processing instruction 
> >>>>"<?META mutexes="test"?> or <?MUTEX
> >>>>names="xxx,yyy"?> (this is more 
> >>>>readable, but more complex too - most likely it will
> >>>>require two level 
> >>>>Map of meta attributes) and keeps them in an
> >>>>identity hash map within 
> >>>>project. Theoretically any object should be able to
> >>>>have a meta 
> >>>>information attached, so during real task creation
> >>>>the hash map should 
> >>>>be updated with real reference. What do you think?
> >>>>
> >>>>- Alexey.
> >>>>
> >>>>
> >>>>   
> >>>>
> >>>>        
> >>>>
> >>>-----------------------------------------------------------
> ----------
> >>> 
> >>>
> >>>      
> >>>
> >>>>To unsubscribe, e-mail:
> >>>>dev-unsubscribe@ant.apache.org
> >>>>For additional commands, e-mail:
> >>>>dev-help@ant.apache.org
> >>>>
> >>>>
> >>>>   
> >>>>
> >>>>        
> >>>>
> >>>__________________________________________________
> >>>Do You Yahoo!?
> >>>Tired of spam?  Yahoo! Mail has the best spam protection around 
> >>>http://mail.yahoo.com
> >>>
> >>>-----------------------------------------------------------
> ----------
> >>>To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
> >>>For additional commands, e-mail: dev-help@ant.apache.org
> >>> 
> >>>
> >>>      
> >>>
> >>--
> >>--------------------------------------------------------------
> >>----------
> >>/ Alexey N. Solofnenko
> >>MDL Information Systems, Inc.
> >>work: 510-357-2222x1726
> >>/
> >>
> >>------------------------------------------------------------
> ---------
> >>To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
> >>For additional commands, e-mail: dev-help@ant.apache.org
> >>
> >>
> >>    
> >>
> >
> >---------------------------------------------------------------------
> >To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
> >For additional commands, e-mail: dev-help@ant.apache.org
> >  
> >
> 
> -- 
> --------------------------------------------------------------
> ----------
> / Alexey N. Solofnenko
> MDL Information Systems, Inc.
> work: 510-357-2222x1726
> /
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
> For additional commands, e-mail: dev-help@ant.apache.org
> 
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org


Mime
View raw message