Mailing-List: contact dev-help@ant.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Ant Developers List" <dev@ant.apache.org>
Received-SPF: softfail (athena.apache.org: transitioning domain of
 dridi.boukelmoune@zenika.com does not designate 207.126.144.137 as permitted
 sender)
MIME-Version: 1.0
In-Reply-To: <BLU172-W410C8B2FCACB49CAA55D5FAE490@phx.gbl>
References: 
 <CA+e2WO7VeHzL7t9SXs4xP00jh0Y5uD5VcmDdsaoe9WYWVokF4A@mail.gmail.com>
 <90546E46-3A9C-4AF1-B097-EA1A965D6520@hibnet.org>
 <CA+e2WO7hbVv+By0at9UOj04TcWfosv8C6+A2CuAqSiEtCJihzA@mail.gmail.com>
 <E924F668-FBBB-4F9B-A642-7C0390B961FF@hibnet.org>
 <CABtDKm6GoR+Z+V5PbBZoW04huAFqi_HF5igQiaoJ7S_8WAzWew@mail.gmail.com>
 <BLU172-W410C8B2FCACB49CAA55D5FAE490@phx.gbl>
From: Dridi Boukelmoune <dridi.boukelmoune@zenika.com>
Date: Tue, 27 Aug 2013 08:22:10 +0200
Message-ID: 
 <CABtDKm657i1wed7V4wV8ySxZc0SwT-ZRp5kJYj0ndXaE6uix2g@mail.gmail.com>
Subject: Re: Easyant - Plugin conflict management
To: Ant Developers List <dev@ant.apache.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hi Martin,

I believe one of my comments must have offended you and maybe others.
This was a reference to the shitty-maven-plugin, which made me laugh
the first time I saw it. For anything other than Maven I would have
said something along "and does a poor job". I should have written
"shitty[x]" and put my reference to make it obvious, I suppose only
plugin writers tend to know this one.

On Tue, Aug 27, 2013 at 12:04 AM, Martin Gainty <mgainty@hotmail.com> wrote=
:
> you never looked at plugin-management

Yes I know, use and encourage others to use it at work.

> understanding what you're criticising before you write a commentary will =
lend veracity to your statements

You are right about understanding, this is why I said "I think I've
observed". However I use plugin-management to make modules
dependencies converge (in a multi-module project or let's say
company-wide with a common parent pom). So that for instance all my
jar modules use the same version/configuration of the *implicitly
declared* compiler plugin.

But in my comment I'm not talking about plugin-management, I'm talking
about plugin dependencies.

>From the first few seconds of a "mvn -X clean verify" build:
[DEBUG] Populating class realm
plugin>org.apache.maven.plugins:maven-clean-plugin:2.4.1
[DEBUG]   Included: org.apache.maven.plugins:maven-clean-plugin:jar:2.4.1
[DEBUG]   Included: org.codehaus.plexus:plexus-utils:jar:2.0.5
[DEBUG]   Excluded: org.apache.maven:maven-plugin-api:jar:2.0.6
[DEBUG] Configuring mojo
org.apache.maven.plugins:maven-clean-plugin:2.4.1:clean from plugin
realm ClassRealm[plugin>org.apache.maven.plugins:maven-clean-plugin:2.4.1,
parent: sun.misc.Launcher$AppClassLoader@e1641c0]

[DEBUG] Populating class realm
plugin>org.apache.maven.plugins:maven-surefire-plugin:2.12
[DEBUG]   Included: org.apache.maven.plugins:maven-surefire-plugin:jar:2.12
[DEBUG]   Included: org.apache.maven.surefire:surefire-booter:jar:2.12
[DEBUG]   Included: org.apache.maven.surefire:surefire-api:jar:2.12
[DEBUG]   Included: org.apache.maven.surefire:maven-surefire-common:jar:2.1=
2
[DEBUG]   Included:
org.apache.maven.shared:maven-common-artifact-filters:jar:1.3
[DEBUG]   Included: org.codehaus.plexus:plexus-utils:jar:3.0
[DEBUG]   Included: org.apache.maven.reporting:maven-reporting-api:jar:2.0.=
9
[DEBUG] Configuring mojo
org.apache.maven.plugins:maven-surefire-plugin:2.12:test from plugin
realm ClassRealm[plugin>org.apache.maven.plugins:maven-surefire-plugin:2.12=
,
parent: sun.misc.Launcher$AppClassLoader@e1641c0]

[DEBUG]   Included: org.codehaus.mojo:clirr-maven-plugin:jar:2.5
[DEBUG]   Included: net.sf.clirr:clirr-core:jar:0.6
[DEBUG]   Included: org.apache.bcel:bcel:jar:5.2
[DEBUG]   Included: jakarta-regexp:jakarta-regexp:jar:1.4
[DEBUG]   Included: org.apache.maven.doxia:doxia-decoration-model:jar:1.0
[DEBUG]   Included: org.apache.maven.doxia:doxia-module-xhtml:jar:1.0
[DEBUG]   Included: org.apache.maven.doxia:doxia-core:jar:1.0
[DEBUG]   Included: org.apache.maven.doxia:doxia-sink-api:jar:1.0
[DEBUG]   Included: org.apache.maven.doxia:doxia-site-renderer:jar:1.0
[DEBUG]   Included: org.codehaus.plexus:plexus-velocity:jar:1.1.7
[DEBUG]   Included: org.apache.velocity:velocity:jar:1.5
[DEBUG]   Included: commons-lang:commons-lang:jar:2.1
[DEBUG]   Included: oro:oro:jar:2.0.8
[DEBUG]   Included: commons-collections:commons-collections:jar:3.2
[DEBUG]   Included: org.apache.maven.doxia:doxia-module-apt:jar:1.0
[DEBUG]   Included: org.apache.maven.doxia:doxia-module-fml:jar:1.0
[DEBUG]   Included: org.apache.maven.doxia:doxia-module-xdoc:jar:1.0
[DEBUG]   Included: org.apache.maven.reporting:maven-reporting-api:jar:2.0.=
6
[DEBUG]   Included: org.codehaus.plexus:plexus-i18n:jar:1.0-beta-6
[DEBUG]   Included: org.codehaus.plexus:plexus-utils:jar:3.0.7
[DEBUG] Configuring mojo
org.codehaus.mojo:clirr-maven-plugin:2.5:check from plugin realm
ClassRealm[plugin>org.codehaus.mojo:clirr-maven-plugin:2.5, parent:
sun.misc.Launcher$AppClassLoader@e1641c0]

I'm omitting logs starting with Excluded (no plexus-utils in those).
In a single build, I get three plugins using three different versions
of plexus-utils (2.0.5, 3.0, 3.0.7). So this is what I meant: is it a
problem for plugins to depend on different versions of other stuff
(plugins or modules) ?

>
> http://maven.apache.org/pom.html#Plugin_Management
> Martin Gainty

I hope I've convinced you I'm not freely flaming on Maven but really
trying to contribute to the thread.

Best Regards,
Dridi

> ______________________________________________
> Verzicht und Vertraulichkeitanmerkung/Note de d=C3=A9ni et de confidentia=
lit=C3=A9
>
> Diese Nachricht ist vertraulich. Sollten Sie nicht der vorgesehene Empfae=
nger sein, so bitten wir hoeflich um eine Mitteilung. Jede unbefugte Weiter=
leitung oder Fertigung einer Kopife ist unzulaessig. Diese Nachricht dient =
lediglich dem Austausch von Informationen und entfaltet keine rechtliche Bi=
ndungswirkung. Aufgrund der leichten Manipulierbarkeit von E-Mails koennen =
wir keine Haftung fuer den Inhalt uebernehmen.
>
> Ce message est confidentiel et peut =C3=AAtre privil=C3=A9gi=C3=A9. Si vo=
us n'=C3=AAtes pas le destinataire pr=C3=A9vu, nous te demandons avec bont=
=C3=A9 que pour satisfaire informez l'exp=C3=A9diteur. N'importe quelle dif=
fusion non autoris=C3=A9e ou la copie de ceci est interdite. Ce message ser=
t =C3=A0 l'information seulement et n'aura pas n'importe quel effet l=C3=A9=
galement obligatoire. =C3=89tant donn=C3=A9 que les email peuvent facilemen=
t =C3=AAtre sujets =C3=A0 la manipulation, nous ne pouvons accepter aucune =
responsabilit=C3=A9 pour le contenu fourni.
>
>
>
>
>> From: dridi.boukelmoune@zenika.com
>> Date: Mon, 26 Aug 2013 20:31:32 +0200
>> Subject: Re: Easyant - Plugin conflict management
>> To: dev@ant.apache.org
>>
>> Hi,
>>
>> This is my first post on this mailing list, I used to lurk in the
>> previous easant ML and even answer sometimes.
>>
>> In this thread I see several points and one of them looks like a
>> non-issue to me. Please forgive my heresy when I use maven as an
>> example.
>>
>> On Tue, Aug 20, 2013 at 6:50 PM, Nicolas Lalev=C3=A9e
>> <nicolas.lalevee@hibnet.org> wrote:
>> >
>> > Le 20 ao=C3=BBt 2013 =C3=A0 14:22, Jean-Louis Boudart <jeanlouis.bouda=
rt@gmail.com> a =C3=A9crit :
>> >
>> >> I've commited changes on trunk with some unit tests if you want to ha=
ve a
>> >> look.
>> >>
>> >> Known limitation :
>> >> * system plugins doesn't participate *yet* to global plugin resolutio=
n
>> >> * plugins classpath created dinamically will all contains the whole
>> >> resolution graph (not really a problem)
>> >>
>> >> Point 1 : this can even affect performance. Invoking resolve process =
one
>> >> time seems much more speed that invoking plugins individually.
>> >
>> > This seems strange to me. Do you have an idea why ?
>>
>> I believe he meant this could *improve* performances by sharing a
>> single global resolve between all plugins (or something like that).
>>
>> >> Point 2 : Even if there is no much activity, i suggest to keep backwa=
rd
>> >> compatibility
>> >
>> > fine for me.
>>
>> Not necessary from my point of view. My only advice is to avoid legacy
>> (or technical debt ;) at such early time.
>>
>> >> Point 3 : By two steps i meant running a global resolve (for all plug=
ins
>> >> and buildtypes including transitive ones). And then have a second cla=
ss
>> >> invoked to perform individual import of ant build file by picking the=
m from
>> >> the ResolveReport.
>> >
>> > ok I see. This make totally sense.
>>
>> This is where I'm lost. Actually the whole issue of dependencies
>> conflicts between plugins.
>>
>> Here is the behavior I think I've observed with Maven:
>> - maven resolves project dependencies and does a shitty job at
>> conflicts resolution
>> - maven resolves plugins dependencies one by one and downloads them lazi=
ly
>> - each plugin is executed with its own classloader, and doesn't care
>> about other plugins dependencies
>> - I can get a dozen versions of the infamous[1] plexus-utils in a single=
 build
>>
>> My point is, given proper isolation, is it a real issue to have
>> different plugins depending on different versions of the same modules
>> ?
>>
>> At the very least, I like the idea of a single global resolve. Even
>> though I understand the benefits of a lazy resolution/download, I
>> don't see this as a problem with today's average bandwidth and disk
>> space to get every thing needed by a build in a single go. I tend to
>> work offline and I hate when I discover in the train that I'm missing
>> a dependency :)
>>
>> >> You can check ResolvePlugins and ImportDeferred class in trunk if you=
 want
>> >> to see concrete stuff.
>> >
>> > I usually do a very quick review of the code to keep me informed of wh=
at is going on and see if nothing bug me, and as always, it seems great. Bu=
t I need to get into this more deeply to see how it really works.
>> >
>> > Nicolas
>> >
>>
>> Best Regards,
>> Dridi
>>
>> [1] it's just that I don't like it :p
>>
>> >>
>> >>
>> >>
>> >>
>> >> 2013/8/17 Nicolas Lalev=C3=A9e <nicolas.lalevee@hibnet.org>
>> >>
>> >>> Long overdue response.
>> >>>
>> >>> Le 3 ao=C3=BBt 2013 =C3=A0 13:33, Jean-Louis Boudart <jeanlouis.boud=
art@gmail.com>
>> >>> a =C3=A9crit :
>> >>>
>> >>>> Hi there,
>> >>>>
>> >>>> It becomes necesssary to manage conflicts between plugins.
>> >>>> This issues has been raised many time and is referenced on jira[1].
>> >>>>
>> >>>> Currently easyant offers import task taking some specific attribute=
s to
>> >>>> resolve a plugin (mainly organisation, module name and revision).
>> >>>> <ea:import org=3D"mycompany" module=3D"myplugin" revision=3D"x.x"/>
>> >>>>
>> >>>> This task will :
>> >>>> * resolve the given plugin or buildtypr
>> >>>> * create a dynamic classpath for this plugin
>> >>>> * expose location of others extra files through properties (ex
>> >>>> checkstyle.xml containing checkstyle rules, this files is shipped i=
n the
>> >>>> plugin). Thoses properties will then be reused by the plugin itself
>> >>>> * import the real ant file (invoke the importTask from ant core und=
er the
>> >>>> hood)
>> >>>>
>> >>>> This task is currently used :
>> >>>> * dynamically by easyant to load system plugins (skeletons for exam=
ple)
>> >>>> * dynamically by easyant when you specify <ea:build> or <ea:plugin>=
 tags
>> >>>> in module.ivy files
>> >>>> * invoked in plugin ant file itselfs
>> >>>> * invoked in module.ant if users has complex needs
>> >>>>
>> >>>> Additionnal there is two "alliases" for this task to import plugins=
 and
>> >>>> buildtype.
>> >>>> <ea:plugin module=3D"compile-java" revision=3D"0.9"/>
>> >>>> <ea:plugin module=3D"build-std-java" revision=3D"0.9"/>
>> >>>> If organisation attribute is not specified on aliases default one w=
ill
>> >>> be
>> >>>> used.
>> >>>>
>> >>>> It does the job but it doesn't handle conflict between plugins.
>> >>>>
>> >>>> Some plugins relies on abstract ones.
>> >>>> Exemple:
>> >>>> package-jar depends on abstract-package, abstract-package depends o=
n
>> >>>> abstract-compile, but compile-java plugin also depends on
>> >>> abstract-compile.
>> >>>> Which versi of abstract-compile will be taken in case both plugins =
load
>> >>>> different version ? The answer is the first one !
>> >>>> This becomes more problematic on buildtypes, as buildtypes loads a =
set of
>> >>>> plugins (including themself others abstract-plugins).
>> >>>>
>> >>>> Ok so now you should have a quick picture of the problem.
>> >>>>
>> >>>> What could be done ?
>> >>>>
>> >>>> We can rely on ivy to describe dependency on plugins. But then how =
could
>> >>> we
>> >>>> know in which order plugins should be loaded ?
>> >>>>
>> >>>> I suggest to introduce a deferred import mechanism.
>> >>>>
>> >>>> We should split responsibility in two distinct steps.
>> >>>> 1 - resolve (based on ivy) the whole graph of plugins and store the
>> >>> resolve
>> >>>> report somewhere as a reusable reference in ant project
>> >>>> 2 - invoke a new import task should import already resolved plugins=
 (the
>> >>>> task could rely on the report stored as a reference in ant project)
>> >>>>
>> >>>> Exemple :
>> >>>> compile java will have an ivy dependency on abstract-compile
>> >>>> <dependency org=3D"org.apache.easyant.plugins" module=3D"abstract-c=
ompile"
>> >>>> revision=3D"1.0"/>
>> >>>>
>> >>>> The compile java ant script will import the resolved plugin
>> >>>> <ea:import-deferred org=3D"org.apache.easyant.plugins"
>> >>>> module=3D"abstract-compile"/>
>> >>>>
>> >>>> Note that i'm not fixed yet with the task name. Any suggestion (eve=
n for
>> >>>> alliases are welcome).
>> >>>>
>> >>>> To maintain backward compatibility i'm in favor of creating new ali=
ases
>> >>>> "import-plugin" and "import-buildtype" instead of reusing existing =
ones.
>> >>>>
>> >>>> People would be able to continue using existing task with known the
>> >>>> limitation (no conflict management on plugins).
>> >>>> This can help if someone wants to load plugins in module.ant after
>> >>> setting
>> >>>> a few properties.
>> >>>>
>> >>>> I also recommend adding a warning on existing task to recommend peo=
ple
>> >>>> using the new import mechanism.
>> >>>>
>> >>>> What do you think ?
>> >>>
>> >>> I see 3 points here.
>> >>>
>> >>> First, managing plugin dependencies: with Ivy, of course, we couldn'=
t
>> >>> agree more :)
>> >>>
>> >>> Then about creating new tasks to keep backward compatibility. I thin=
k we
>> >>> can break backward compatibility. Easyant is not yet 1.0 and I do no=
t see
>> >>> much activity on the user list. I would prefer bugging the current u=
sers
>> >>> than having an error-prone and deprecated task around.
>> >>>
>> >>> Third there is the resolve in two steps. I really like the idea. I a=
m not
>> >>> sure though if we need this in order to bring conflict management in=
 plugin
>> >>> dependencies. And I am not sure how far you are willing to go.
>> >>> Actually this is a larger topic which has bugging me recently. The w=
ay we
>> >>> use the ivy.xml is generally either to tight or insecure. For instan=
ce when
>> >>> using version ranges in an ivy.xml, since the content of the reposit=
ory is
>> >>> expected to change over time, then the resolve may change over time =
since
>> >>> new versions might fit the range. Range makes things unreliable over=
 time,
>> >>> so often I restrict myself to not use any range. But it's kind of a =
shame
>> >>> to not use ranges in a dependency manager.
>> >>> I continued experimenting with the OSGi mapping in Ivy. And the OSGi
>> >>> version semantics are very loose. It is because it represents what v=
ersions
>> >>> of software it is compatible with, not the versions we will be using=
 in
>> >>> your specific unit test or application. So relying on OSGi manifest =
to
>> >>> resolve dependencies is not safe at all. That's why I implemented th=
e
>> >>> fixdeps [1] task. From a very loose specification of the dependencie=
s, it
>> >>> will produce an ivy.xml which is very tight and secure.
>> >>> Then, when adding dependencies to a project, we edit the
>> >>> ivy-specification.xml and run fixdeps. The build then relies on the
>> >>> produced ivy.xml. A nice side effect is that since there is only non
>> >>> transitive fixed dependencies to resolve, resolve is fast: either th=
e
>> >>> module is here either it's not. And with proper caching, everything =
works
>> >>> with the filesystem.
>> >>> But, as wrote above, I'm not sure if that's why you suggested the tw=
o
>> >>> steps resolve.
>> >>>
>> >>> Nicolas
>> >>>
>> >>> [1] http://ant.apache.org/ivy/history/trunk/use/fixdeps.html
>> >>>
>> >>>
>> >>> --------------------------------------------------------------------=
-
>> >>> To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
>> >>> For additional commands, e-mail: dev-help@ant.apache.org
>> >>>
>> >>>
>> >>
>> >>
>> >> --
>> >> Jean Louis Boudart
>> >> Independent consultant
>> >> Apache EasyAnt commiter http://ant.apache.org/easyant/
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
>> > For additional commands, e-mail: dev-help@ant.apache.org
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
>> For additional commands, e-mail: dev-help@ant.apache.org
>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org