ant-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dridi Boukelmoune <dridi.boukelmo...@zenika.com>
Subject Re: Easyant - Plugin conflict management
Date Tue, 27 Aug 2013 09:14:40 GMT
I also forgot one comparison.

>From my understaning, please correct me if I'm wrong:
- maven plugins are bound to phases, lifecycles specify the order of the phases
- easyant plugins are bound to extension points, the order derives
from dependencies between the extension points
- maven solves (plugin) dependencies through a chain of inheritance
- maven fills by default the lifecycle with goals depending on the
module packaging
- easyant can fill the lifecycle with build-type plugins (to make things easier)

Now Jean-Louis said:
>>> >>>> Some plugins relies on abstract ones.
>>> >>>> Exemple:
>>> >>>> package-jar depends on abstract-package, abstract-package depends
on
>>> >>>> abstract-compile, but compile-java plugin also depends on
>>> >>> abstract-compile.
>>> >>>> Which versi of abstract-compile will be taken in case both plugins
load
>>> >>>> different version ? The answer is the first one !
>>> >>>> This becomes more problematic on buildtypes, as buildtypes loads
a set of
>>> >>>> plugins (including themself others abstract-plugins).

What bugs me, is the fact that changing package-jar's version could
change my compile-java's version. This is just a caricature, but for
me the former creates a zip while the latter invokes javac ; they
should be unrelated. If both package-jar and compile-java with
diverging versions conflict on a dependency, they should each get the
one they need. I take the plexus-utils dependency as an example,
because a lot of maven plugins use it. So when I change my
maven-package-plugin version, it has no effect AFAIK on my
maven-compiler-plugin. Maven makes it hard to customize the lifecycle,
so I've never met someone willing to do so.

My point of view in a nutshell:
- "build-type" plugins help define easily the lifecycle
- "regular" plugins should not drag other plugins

Now what happens when you have several build-types (eg. a polyglot project) ?
I'd tend to think that a build-type should override previous
build-types (eg. package-jar needed by both java and scala builds).

Best Regards,
Dridi

On Tue, Aug 27, 2013 at 8:22 AM, Dridi Boukelmoune
<dridi.boukelmoune@zenika.com> wrote:
> Hi Martin,
>
> I believe one of my comments must have offended you and maybe others.
> This was a reference to the shitty-maven-plugin, which made me laugh
> the first time I saw it. For anything other than Maven I would have
> said something along "and does a poor job". I should have written
> "shitty[x]" and put my reference to make it obvious, I suppose only
> plugin writers tend to know this one.
>
> On Tue, Aug 27, 2013 at 12:04 AM, Martin Gainty <mgainty@hotmail.com> wrote:
>> you never looked at plugin-management
>
> Yes I know, use and encourage others to use it at work.
>
>> understanding what you're criticising before you write a commentary will lend veracity
to your statements
>
> You are right about understanding, this is why I said "I think I've
> observed". However I use plugin-management to make modules
> dependencies converge (in a multi-module project or let's say
> company-wide with a common parent pom). So that for instance all my
> jar modules use the same version/configuration of the *implicitly
> declared* compiler plugin.
>
> But in my comment I'm not talking about plugin-management, I'm talking
> about plugin dependencies.
>
> From the first few seconds of a "mvn -X clean verify" build:
> [DEBUG] Populating class realm
> plugin>org.apache.maven.plugins:maven-clean-plugin:2.4.1
> [DEBUG]   Included: org.apache.maven.plugins:maven-clean-plugin:jar:2.4.1
> [DEBUG]   Included: org.codehaus.plexus:plexus-utils:jar:2.0.5
> [DEBUG]   Excluded: org.apache.maven:maven-plugin-api:jar:2.0.6
> [DEBUG] Configuring mojo
> org.apache.maven.plugins:maven-clean-plugin:2.4.1:clean from plugin
> realm ClassRealm[plugin>org.apache.maven.plugins:maven-clean-plugin:2.4.1,
> parent: sun.misc.Launcher$AppClassLoader@e1641c0]
>
> [DEBUG] Populating class realm
> plugin>org.apache.maven.plugins:maven-surefire-plugin:2.12
> [DEBUG]   Included: org.apache.maven.plugins:maven-surefire-plugin:jar:2.12
> [DEBUG]   Included: org.apache.maven.surefire:surefire-booter:jar:2.12
> [DEBUG]   Included: org.apache.maven.surefire:surefire-api:jar:2.12
> [DEBUG]   Included: org.apache.maven.surefire:maven-surefire-common:jar:2.12
> [DEBUG]   Included:
> org.apache.maven.shared:maven-common-artifact-filters:jar:1.3
> [DEBUG]   Included: org.codehaus.plexus:plexus-utils:jar:3.0
> [DEBUG]   Included: org.apache.maven.reporting:maven-reporting-api:jar:2.0.9
> [DEBUG] Configuring mojo
> org.apache.maven.plugins:maven-surefire-plugin:2.12:test from plugin
> realm ClassRealm[plugin>org.apache.maven.plugins:maven-surefire-plugin:2.12,
> parent: sun.misc.Launcher$AppClassLoader@e1641c0]
>
> [DEBUG]   Included: org.codehaus.mojo:clirr-maven-plugin:jar:2.5
> [DEBUG]   Included: net.sf.clirr:clirr-core:jar:0.6
> [DEBUG]   Included: org.apache.bcel:bcel:jar:5.2
> [DEBUG]   Included: jakarta-regexp:jakarta-regexp:jar:1.4
> [DEBUG]   Included: org.apache.maven.doxia:doxia-decoration-model:jar:1.0
> [DEBUG]   Included: org.apache.maven.doxia:doxia-module-xhtml:jar:1.0
> [DEBUG]   Included: org.apache.maven.doxia:doxia-core:jar:1.0
> [DEBUG]   Included: org.apache.maven.doxia:doxia-sink-api:jar:1.0
> [DEBUG]   Included: org.apache.maven.doxia:doxia-site-renderer:jar:1.0
> [DEBUG]   Included: org.codehaus.plexus:plexus-velocity:jar:1.1.7
> [DEBUG]   Included: org.apache.velocity:velocity:jar:1.5
> [DEBUG]   Included: commons-lang:commons-lang:jar:2.1
> [DEBUG]   Included: oro:oro:jar:2.0.8
> [DEBUG]   Included: commons-collections:commons-collections:jar:3.2
> [DEBUG]   Included: org.apache.maven.doxia:doxia-module-apt:jar:1.0
> [DEBUG]   Included: org.apache.maven.doxia:doxia-module-fml:jar:1.0
> [DEBUG]   Included: org.apache.maven.doxia:doxia-module-xdoc:jar:1.0
> [DEBUG]   Included: org.apache.maven.reporting:maven-reporting-api:jar:2.0.6
> [DEBUG]   Included: org.codehaus.plexus:plexus-i18n:jar:1.0-beta-6
> [DEBUG]   Included: org.codehaus.plexus:plexus-utils:jar:3.0.7
> [DEBUG] Configuring mojo
> org.codehaus.mojo:clirr-maven-plugin:2.5:check from plugin realm
> ClassRealm[plugin>org.codehaus.mojo:clirr-maven-plugin:2.5, parent:
> sun.misc.Launcher$AppClassLoader@e1641c0]
>
> I'm omitting logs starting with Excluded (no plexus-utils in those).
> In a single build, I get three plugins using three different versions
> of plexus-utils (2.0.5, 3.0, 3.0.7). So this is what I meant: is it a
> problem for plugins to depend on different versions of other stuff
> (plugins or modules) ?
>
>>
>> http://maven.apache.org/pom.html#Plugin_Management
>> Martin Gainty
>
> I hope I've convinced you I'm not freely flaming on Maven but really
> trying to contribute to the thread.
>
> Best Regards,
> Dridi
>
>> ______________________________________________
>> Verzicht und Vertraulichkeitanmerkung/Note de déni et de confidentialité
>>
>> Diese Nachricht ist vertraulich. Sollten Sie nicht der vorgesehene Empfaenger sein,
so bitten wir hoeflich um eine Mitteilung. Jede unbefugte Weiterleitung oder Fertigung einer
Kopife ist unzulaessig. Diese Nachricht dient lediglich dem Austausch von Informationen und
entfaltet keine rechtliche Bindungswirkung. Aufgrund der leichten Manipulierbarkeit von E-Mails
koennen wir keine Haftung fuer den Inhalt uebernehmen.
>>
>> Ce message est confidentiel et peut être privilégié. Si vous n'êtes pas le destinataire
prévu, nous te demandons avec bonté que pour satisfaire informez l'expéditeur. N'importe
quelle diffusion non autorisée ou la copie de ceci est interdite. Ce message sert à l'information
seulement et n'aura pas n'importe quel effet légalement obligatoire. Étant donné que les
email peuvent facilement être sujets à la manipulation, nous ne pouvons accepter aucune
responsabilité pour le contenu fourni.
>>
>>
>>
>>
>>> From: dridi.boukelmoune@zenika.com
>>> Date: Mon, 26 Aug 2013 20:31:32 +0200
>>> Subject: Re: Easyant - Plugin conflict management
>>> To: dev@ant.apache.org
>>>
>>> Hi,
>>>
>>> This is my first post on this mailing list, I used to lurk in the
>>> previous easant ML and even answer sometimes.
>>>
>>> In this thread I see several points and one of them looks like a
>>> non-issue to me. Please forgive my heresy when I use maven as an
>>> example.
>>>
>>> On Tue, Aug 20, 2013 at 6:50 PM, Nicolas Lalevée
>>> <nicolas.lalevee@hibnet.org> wrote:
>>> >
>>> > Le 20 août 2013 à 14:22, Jean-Louis Boudart <jeanlouis.boudart@gmail.com>
a écrit :
>>> >
>>> >> I've commited changes on trunk with some unit tests if you want to have
a
>>> >> look.
>>> >>
>>> >> Known limitation :
>>> >> * system plugins doesn't participate *yet* to global plugin resolution
>>> >> * plugins classpath created dinamically will all contains the whole
>>> >> resolution graph (not really a problem)
>>> >>
>>> >> Point 1 : this can even affect performance. Invoking resolve process
one
>>> >> time seems much more speed that invoking plugins individually.
>>> >
>>> > This seems strange to me. Do you have an idea why ?
>>>
>>> I believe he meant this could *improve* performances by sharing a
>>> single global resolve between all plugins (or something like that).
>>>
>>> >> Point 2 : Even if there is no much activity, i suggest to keep backward
>>> >> compatibility
>>> >
>>> > fine for me.
>>>
>>> Not necessary from my point of view. My only advice is to avoid legacy
>>> (or technical debt ;) at such early time.
>>>
>>> >> Point 3 : By two steps i meant running a global resolve (for all plugins
>>> >> and buildtypes including transitive ones). And then have a second class
>>> >> invoked to perform individual import of ant build file by picking them
from
>>> >> the ResolveReport.
>>> >
>>> > ok I see. This make totally sense.
>>>
>>> This is where I'm lost. Actually the whole issue of dependencies
>>> conflicts between plugins.
>>>
>>> Here is the behavior I think I've observed with Maven:
>>> - maven resolves project dependencies and does a shitty job at
>>> conflicts resolution
>>> - maven resolves plugins dependencies one by one and downloads them lazily
>>> - each plugin is executed with its own classloader, and doesn't care
>>> about other plugins dependencies
>>> - I can get a dozen versions of the infamous[1] plexus-utils in a single build
>>>
>>> My point is, given proper isolation, is it a real issue to have
>>> different plugins depending on different versions of the same modules
>>> ?
>>>
>>> At the very least, I like the idea of a single global resolve. Even
>>> though I understand the benefits of a lazy resolution/download, I
>>> don't see this as a problem with today's average bandwidth and disk
>>> space to get every thing needed by a build in a single go. I tend to
>>> work offline and I hate when I discover in the train that I'm missing
>>> a dependency :)
>>>
>>> >> You can check ResolvePlugins and ImportDeferred class in trunk if you
want
>>> >> to see concrete stuff.
>>> >
>>> > I usually do a very quick review of the code to keep me informed of what
is going on and see if nothing bug me, and as always, it seems great. But I need to get into
this more deeply to see how it really works.
>>> >
>>> > Nicolas
>>> >
>>>
>>> Best Regards,
>>> Dridi
>>>
>>> [1] it's just that I don't like it :p
>>>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> 2013/8/17 Nicolas Lalevée <nicolas.lalevee@hibnet.org>
>>> >>
>>> >>> Long overdue response.
>>> >>>
>>> >>> Le 3 août 2013 à 13:33, Jean-Louis Boudart <jeanlouis.boudart@gmail.com>
>>> >>> a écrit :
>>> >>>
>>> >>>> Hi there,
>>> >>>>
>>> >>>> It becomes necesssary to manage conflicts between plugins.
>>> >>>> This issues has been raised many time and is referenced on jira[1].
>>> >>>>
>>> >>>> Currently easyant offers import task taking some specific attributes
to
>>> >>>> resolve a plugin (mainly organisation, module name and revision).
>>> >>>> <ea:import org="mycompany" module="myplugin" revision="x.x"/>
>>> >>>>
>>> >>>> This task will :
>>> >>>> * resolve the given plugin or buildtypr
>>> >>>> * create a dynamic classpath for this plugin
>>> >>>> * expose location of others extra files through properties (ex
>>> >>>> checkstyle.xml containing checkstyle rules, this files is shipped
in the
>>> >>>> plugin). Thoses properties will then be reused by the plugin
itself
>>> >>>> * import the real ant file (invoke the importTask from ant core
under the
>>> >>>> hood)
>>> >>>>
>>> >>>> This task is currently used :
>>> >>>> * dynamically by easyant to load system plugins (skeletons for
example)
>>> >>>> * dynamically by easyant when you specify <ea:build> or
<ea:plugin> tags
>>> >>>> in module.ivy files
>>> >>>> * invoked in plugin ant file itselfs
>>> >>>> * invoked in module.ant if users has complex needs
>>> >>>>
>>> >>>> Additionnal there is two "alliases" for this task to import
plugins and
>>> >>>> buildtype.
>>> >>>> <ea:plugin module="compile-java" revision="0.9"/>
>>> >>>> <ea:plugin module="build-std-java" revision="0.9"/>
>>> >>>> If organisation attribute is not specified on aliases default
one will
>>> >>> be
>>> >>>> used.
>>> >>>>
>>> >>>> It does the job but it doesn't handle conflict between plugins.
>>> >>>>
>>> >>>> Some plugins relies on abstract ones.
>>> >>>> Exemple:
>>> >>>> package-jar depends on abstract-package, abstract-package depends
on
>>> >>>> abstract-compile, but compile-java plugin also depends on
>>> >>> abstract-compile.
>>> >>>> Which versi of abstract-compile will be taken in case both plugins
load
>>> >>>> different version ? The answer is the first one !
>>> >>>> This becomes more problematic on buildtypes, as buildtypes loads
a set of
>>> >>>> plugins (including themself others abstract-plugins).
>>> >>>>
>>> >>>> Ok so now you should have a quick picture of the problem.
>>> >>>>
>>> >>>> What could be done ?
>>> >>>>
>>> >>>> We can rely on ivy to describe dependency on plugins. But then
how could
>>> >>> we
>>> >>>> know in which order plugins should be loaded ?
>>> >>>>
>>> >>>> I suggest to introduce a deferred import mechanism.
>>> >>>>
>>> >>>> We should split responsibility in two distinct steps.
>>> >>>> 1 - resolve (based on ivy) the whole graph of plugins and store
the
>>> >>> resolve
>>> >>>> report somewhere as a reusable reference in ant project
>>> >>>> 2 - invoke a new import task should import already resolved
plugins (the
>>> >>>> task could rely on the report stored as a reference in ant project)
>>> >>>>
>>> >>>> Exemple :
>>> >>>> compile java will have an ivy dependency on abstract-compile
>>> >>>> <dependency org="org.apache.easyant.plugins" module="abstract-compile"
>>> >>>> revision="1.0"/>
>>> >>>>
>>> >>>> The compile java ant script will import the resolved plugin
>>> >>>> <ea:import-deferred org="org.apache.easyant.plugins"
>>> >>>> module="abstract-compile"/>
>>> >>>>
>>> >>>> Note that i'm not fixed yet with the task name. Any suggestion
(even for
>>> >>>> alliases are welcome).
>>> >>>>
>>> >>>> To maintain backward compatibility i'm in favor of creating
new aliases
>>> >>>> "import-plugin" and "import-buildtype" instead of reusing existing
ones.
>>> >>>>
>>> >>>> People would be able to continue using existing task with known
the
>>> >>>> limitation (no conflict management on plugins).
>>> >>>> This can help if someone wants to load plugins in module.ant
after
>>> >>> setting
>>> >>>> a few properties.
>>> >>>>
>>> >>>> I also recommend adding a warning on existing task to recommend
people
>>> >>>> using the new import mechanism.
>>> >>>>
>>> >>>> What do you think ?
>>> >>>
>>> >>> I see 3 points here.
>>> >>>
>>> >>> First, managing plugin dependencies: with Ivy, of course, we couldn't
>>> >>> agree more :)
>>> >>>
>>> >>> Then about creating new tasks to keep backward compatibility. I
think we
>>> >>> can break backward compatibility. Easyant is not yet 1.0 and I do
not see
>>> >>> much activity on the user list. I would prefer bugging the current
users
>>> >>> than having an error-prone and deprecated task around.
>>> >>>
>>> >>> Third there is the resolve in two steps. I really like the idea.
I am not
>>> >>> sure though if we need this in order to bring conflict management
in plugin
>>> >>> dependencies. And I am not sure how far you are willing to go.
>>> >>> Actually this is a larger topic which has bugging me recently. The
way we
>>> >>> use the ivy.xml is generally either to tight or insecure. For instance
when
>>> >>> using version ranges in an ivy.xml, since the content of the repository
is
>>> >>> expected to change over time, then the resolve may change over time
since
>>> >>> new versions might fit the range. Range makes things unreliable
over time,
>>> >>> so often I restrict myself to not use any range. But it's kind of
a shame
>>> >>> to not use ranges in a dependency manager.
>>> >>> I continued experimenting with the OSGi mapping in Ivy. And the
OSGi
>>> >>> version semantics are very loose. It is because it represents what
versions
>>> >>> of software it is compatible with, not the versions we will be using
in
>>> >>> your specific unit test or application. So relying on OSGi manifest
to
>>> >>> resolve dependencies is not safe at all. That's why I implemented
the
>>> >>> fixdeps [1] task. From a very loose specification of the dependencies,
it
>>> >>> will produce an ivy.xml which is very tight and secure.
>>> >>> Then, when adding dependencies to a project, we edit the
>>> >>> ivy-specification.xml and run fixdeps. The build then relies on
the
>>> >>> produced ivy.xml. A nice side effect is that since there is only
non
>>> >>> transitive fixed dependencies to resolve, resolve is fast: either
the
>>> >>> module is here either it's not. And with proper caching, everything
works
>>> >>> with the filesystem.
>>> >>> But, as wrote above, I'm not sure if that's why you suggested the
two
>>> >>> steps resolve.
>>> >>>
>>> >>> Nicolas
>>> >>>
>>> >>> [1] http://ant.apache.org/ivy/history/trunk/use/fixdeps.html
>>> >>>
>>> >>>
>>> >>> ---------------------------------------------------------------------
>>> >>> To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
>>> >>> For additional commands, e-mail: dev-help@ant.apache.org
>>> >>>
>>> >>>
>>> >>
>>> >>
>>> >> --
>>> >> Jean Louis Boudart
>>> >> Independent consultant
>>> >> Apache EasyAnt commiter http://ant.apache.org/easyant/
>>> >
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
>>> > For additional commands, e-mail: dev-help@ant.apache.org
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
>>> For additional commands, e-mail: dev-help@ant.apache.org
>>>
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org


Mime
View raw message