ant-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicolas Lalevée <nicolas.lale...@hibnet.org>
Subject Re: Seek Feedback on ANT/IVY enhancement for incremental multi-module build (a buildlist alternative)
Date Wed, 12 Oct 2011 16:47:15 GMT

Le 12 oct. 2011 à 18:06, Richard Mauri a écrit :

> Please comment on a couple enhancements relating to performing an
> optimized multi-module incremental build.
> 
> I am driving an effort at Ariba, Inc. to use these approaches to
> optimize their large product build system. The work is not yet complete,
> and I've been given a green light to contribute this work to Apache
> community if it makes sense.
> 
> 
> 1. Given a set of identified leaf modules that were known to have
> changed, perform an optimal transitive rebuild only if a performing a
> binary compatibility check of the leaf's artifacts are incompatible with
> respect to prior published artifacts.
> 
> This implies that the build/analysis would be performed while traversing
> the dependency DAG - not pre-computing a list as is done with
> ivy:buildlist.
> 
> The analysis code should be pluggable, and certainly the binary
> compatibility checker plug-in would be available.
> I have a BCEL based dependency checker that implements the JLS binary
> compatibility specification (Ch 13)
> 
> 2. Another plug-in could be a metadata validator, which would compare
> the declared ivy.xml dependencies against the external dependencies as
> discovered by byte code analysis of just product artifacts. The point of
> the validator is to detect that correct first-order dependencies are
> declared. This helps ensure that first-order dependencies are declared -
> for this to work, an index that maps Java package name to defining
> modules is consulted. If there is an ivy dependency that is not in the
> discovered set of available defining modules there is an option to fail
> the build. The validator also leverages BCEL to get the external
> dependencies.
> 
> 3. Another plug-in is an indexer tool that incrementally updates the
> java to owning modules index mentioned above.
> This also leverages BCEL to report what packages the artifact publishes.
> 
> 
> So to recap:
> 
> While actually doing the DFS in the dependency DAG, identify leaf
> components and call out to a module that:
> a) performs the module build
> b) calls out the analysis modules that:
> 
>   b1) checks binary compatibility of latest artifact vs. prior artifact
> and sets a flag that influences whether the next module in the DAG
> traversal needs to be transitively rebuilt.
> 
>   b2) update the Java pkg to owning module index again by leveraging
> BCEL to keep
> 
>   b3) validates the artifact's external first-order dependencies from
> byte code analysis match what's declared in ivy.xml - optionally fail
> the build.
> 
> 
> I think that captures the gist of it.
> 
> I'm having a hard time explaining to the VP's why nobody else has
> thought or implemented this.

probably some have thought of this, but the time to actually implement is often a hard line
to cross.

> Any helpful feedback on that is
> appreciated.

I have no particular feedback to give apart that this looks very interesting !

Your presentation of the integration into Ivy makes totally sense. No deep patching in Ivy
but a new feature: a DAG dependency traverser based on a set of ModuleDescriptors. Just that
it would be fine for me to integrate it into Ivy. If you also provide some plugin/hooks like
the ones you need, it would be awesome.

Nicolas


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org


Mime
View raw message