ant-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dridi Boukelmoune <dridi.boukelmo...@zenika.com>
Subject Re: Easyant - Plugin conflict management
Date Mon, 26 Aug 2013 18:31:32 GMT
Hi,

This is my first post on this mailing list, I used to lurk in the
previous easant ML and even answer sometimes.

In this thread I see several points and one of them looks like a
non-issue to me. Please forgive my heresy when I use maven as an
example.

On Tue, Aug 20, 2013 at 6:50 PM, Nicolas Lalevée
<nicolas.lalevee@hibnet.org> wrote:
>
> Le 20 août 2013 à 14:22, Jean-Louis Boudart <jeanlouis.boudart@gmail.com> a écrit
:
>
>> I've commited changes on trunk with some unit tests if you want to have a
>> look.
>>
>> Known limitation :
>> * system plugins doesn't participate *yet* to global plugin resolution
>> * plugins classpath created dinamically will all contains the whole
>> resolution graph (not really a problem)
>>
>> Point 1 : this can even affect performance. Invoking resolve process one
>> time seems much more speed that invoking plugins individually.
>
> This seems strange to me. Do you have an idea why ?

I believe he meant this could *improve* performances by sharing a
single global resolve between all plugins (or something like that).

>> Point 2 : Even if there is no much activity, i suggest to keep backward
>> compatibility
>
> fine for me.

Not necessary from my point of view. My only advice is to avoid legacy
(or technical debt ;) at such early time.

>> Point 3 : By two steps i meant running a global resolve (for all plugins
>> and buildtypes including transitive ones). And then have a second class
>> invoked to perform individual import of ant build file by picking them from
>> the ResolveReport.
>
> ok I see. This make totally sense.

This is where I'm lost. Actually the whole issue of dependencies
conflicts between plugins.

Here is the behavior I think I've observed with Maven:
- maven resolves project dependencies and does a shitty job at
conflicts resolution
- maven resolves plugins dependencies one by one and downloads them lazily
- each plugin is executed with its own classloader, and doesn't care
about other plugins dependencies
- I can get a dozen versions of the infamous[1] plexus-utils in a single build

My point is, given proper isolation, is it a real issue to have
different plugins depending on different versions of the same modules
?

At the very least, I like the idea of a single global resolve. Even
though I understand the benefits of a lazy resolution/download, I
don't see this as a problem with today's average bandwidth and disk
space to get every thing needed by a build in a single go. I tend to
work offline and I hate when I discover in the train that I'm missing
a dependency :)

>> You can check  ResolvePlugins and ImportDeferred class in trunk if you want
>> to see concrete stuff.
>
> I usually do a very quick review of the code to keep me informed of what is going on
and see if nothing bug me, and as always, it seems great. But I need to get into this more
deeply to see how it really works.
>
> Nicolas
>

Best Regards,
Dridi

[1] it's just that I don't like it :p

>>
>>
>>
>>
>> 2013/8/17 Nicolas Lalevée <nicolas.lalevee@hibnet.org>
>>
>>> Long overdue response.
>>>
>>> Le 3 août 2013 à 13:33, Jean-Louis Boudart <jeanlouis.boudart@gmail.com>
>>> a écrit :
>>>
>>>> Hi there,
>>>>
>>>> It becomes necesssary to manage conflicts between plugins.
>>>> This issues has been raised many time and is referenced on jira[1].
>>>>
>>>> Currently easyant offers import task taking some specific attributes to
>>>> resolve a plugin (mainly organisation, module name and revision).
>>>> <ea:import org="mycompany" module="myplugin" revision="x.x"/>
>>>>
>>>> This task will :
>>>> * resolve the given plugin or buildtypr
>>>> * create a dynamic classpath for this plugin
>>>> * expose location of others extra files through properties (ex
>>>> checkstyle.xml containing checkstyle rules, this files is shipped in the
>>>> plugin). Thoses properties will then be reused by the plugin itself
>>>> * import the real ant file (invoke the importTask from ant core under the
>>>> hood)
>>>>
>>>> This task is currently used :
>>>> * dynamically by easyant to load system plugins (skeletons for example)
>>>> * dynamically by easyant when you specify <ea:build> or <ea:plugin>
tags
>>>> in module.ivy files
>>>> * invoked in plugin ant file itselfs
>>>> * invoked in module.ant if users has complex needs
>>>>
>>>> Additionnal there is two "alliases" for this task to import plugins and
>>>> buildtype.
>>>> <ea:plugin module="compile-java" revision="0.9"/>
>>>> <ea:plugin module="build-std-java" revision="0.9"/>
>>>> If organisation attribute is not specified on aliases  default one will
>>> be
>>>> used.
>>>>
>>>> It does the job but it doesn't handle conflict between plugins.
>>>>
>>>> Some plugins relies on abstract ones.
>>>> Exemple:
>>>> package-jar depends on abstract-package, abstract-package depends on
>>>> abstract-compile, but compile-java plugin also depends on
>>> abstract-compile.
>>>> Which versi of abstract-compile will be taken in case both plugins load
>>>> different version ? The answer is the first one !
>>>> This becomes more problematic on buildtypes, as buildtypes loads a set of
>>>> plugins (including themself others abstract-plugins).
>>>>
>>>> Ok so now you should have a quick picture of the problem.
>>>>
>>>> What could be done  ?
>>>>
>>>> We can rely on ivy to describe dependency on plugins. But then how could
>>> we
>>>> know in which order plugins should be loaded ?
>>>>
>>>> I suggest to introduce a deferred import mechanism.
>>>>
>>>> We should split responsibility in two distinct steps.
>>>> 1 - resolve (based on ivy) the whole graph of plugins and store the
>>> resolve
>>>> report somewhere as a reusable reference in ant project
>>>> 2 - invoke a new import task should import already resolved plugins (the
>>>> task could rely on the report stored as a reference in ant project)
>>>>
>>>> Exemple :
>>>> compile java will have an ivy dependency on abstract-compile
>>>> <dependency org="org.apache.easyant.plugins" module="abstract-compile"
>>>> revision="1.0"/>
>>>>
>>>> The compile java ant script will import the resolved plugin
>>>> <ea:import-deferred org="org.apache.easyant.plugins"
>>>> module="abstract-compile"/>
>>>>
>>>> Note that i'm not fixed yet with the task name. Any suggestion (even for
>>>> alliases are welcome).
>>>>
>>>> To maintain backward compatibility i'm in favor of creating new aliases
>>>> "import-plugin" and "import-buildtype" instead of reusing existing ones.
>>>>
>>>> People would be able to continue using existing task with known the
>>>> limitation (no conflict management on plugins).
>>>> This can help if someone wants to load plugins in module.ant after
>>> setting
>>>> a few properties.
>>>>
>>>> I also recommend adding a warning on existing task to recommend people
>>>> using the new import mechanism.
>>>>
>>>> What do you think ?
>>>
>>> I see 3 points here.
>>>
>>> First, managing plugin dependencies: with Ivy, of course, we couldn't
>>> agree more :)
>>>
>>> Then about creating new tasks to keep backward compatibility. I think we
>>> can break backward compatibility. Easyant is not yet 1.0 and I do not see
>>> much activity on the user list. I would prefer bugging the current users
>>> than having an error-prone and deprecated task around.
>>>
>>> Third there is the resolve in two steps. I really like the idea. I am not
>>> sure though if we need this in order to bring conflict management in plugin
>>> dependencies. And I am not sure how far you are willing to go.
>>> Actually this is a larger topic which has bugging me recently. The way we
>>> use the ivy.xml is generally either to tight or insecure. For instance when
>>> using version ranges in an ivy.xml, since the content of the repository is
>>> expected to change over time, then the resolve may change over time since
>>> new versions might fit the range. Range makes things unreliable over time,
>>> so often I restrict myself to not use any range. But it's kind of a shame
>>> to not use ranges in a dependency manager.
>>> I continued experimenting with the OSGi mapping in Ivy. And the OSGi
>>> version semantics are very loose. It is because it represents what versions
>>> of software it is compatible with, not the versions we will be using in
>>> your specific unit test or application. So relying on OSGi manifest to
>>> resolve dependencies is not safe at all. That's why I implemented the
>>> fixdeps [1] task. From a very loose specification of the dependencies, it
>>> will produce an ivy.xml which is very tight and secure.
>>> Then, when adding dependencies to a project, we edit the
>>> ivy-specification.xml and run fixdeps. The build then relies on the
>>> produced ivy.xml. A nice side effect is that since there is only non
>>> transitive fixed dependencies to resolve, resolve is fast: either the
>>> module is here either it's not. And with proper caching, everything works
>>> with the filesystem.
>>> But, as wrote above, I'm not sure if that's why you suggested the two
>>> steps resolve.
>>>
>>> Nicolas
>>>
>>> [1] http://ant.apache.org/ivy/history/trunk/use/fixdeps.html
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
>>> For additional commands, e-mail: dev-help@ant.apache.org
>>>
>>>
>>
>>
>> --
>> Jean Louis Boudart
>> Independent consultant
>> Apache EasyAnt commiter http://ant.apache.org/easyant/
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
> For additional commands, e-mail: dev-help@ant.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org


Mime
View raw message