qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robbie Gemmell <robbie.gemm...@gmail.com>
Subject Re: Maven build for the QMF2 tools
Date Fri, 02 May 2014 11:28:13 GMT
On 2 May 2014 11:23, Fraser Adams <fraser.adams@blueyonder.co.uk> wrote:

> I'd be content as long as the "
>
>
> When I release the initial qpid-parent pom in the coming hours it should
> now all 'Just Work'*
>
> "
> caveat is in place :-)
>

Will be soon, releasing it shortly to the ASF repo and then its just a
maven-central sync cycle away.

Note that if you tried the build with the patch I put in the vote email
then you actually already have it installed in your local repo :)


> TBH I've been avoiding touching this on trunk whilst it was in a state of
> flux 'cause I didn't want to add to the carnage, so it'll be good to get
> things onto trunk and hopefully have people playing to help make sure it's
> nicely bedded down and stable again.
>
> Looking at the recent Jiras am I correct that you've made updates so that
> it (the instance on the maven branch) is in sync with the recent Java
> Broker changes?
>
>
Yes, though as Rob said, compilation only. Ensuring it actualy works is
still to come later, which doing this would help facilitate.


> I doubt I'll have a chance to play with this until next weekend at the
> earliest 'cause I've got family stuff planned pretty much all this weekend.
>
> Is there a chance of problems due to me having tried things out on the
> branch previously - in other words is it sensible to blat my .mvn and start
> from scratch or should all the versioning stuff in the various poms make it
> all behave nicely?
>
> Cheers,
> Frase
>
>
I have updated the build itself so that the qmf bits depend on the current
trunk/0.30-SNAPSHOT broker. You should see no issues from having tried
things previously on the branch, as the qmf bits always had 0.30-SNAPSHOT
version themselves on the branch but just depended on the then-current
trunk 0.28-SNAPSHOT broker, which maven will now ignore and use the
0.30-SNAPSHOT artifacts, either grabbing from the snapshots repo or finding
in your local repo from your doing 'mvn install' in the main java tree.

(sidenote: you wouldnt need to nuke the whole ~/.m2 folder in any case if
it was messed up, just removing the subtree org/apache/qpid containing our
artifacts would be sufficient...but you should rarely ever need to do that)



>
>
> On 02/05/14 11:03, Robbie Gemmell wrote:
>
>> Ok, I'd like to merge the maven build for the QMF bits to trunk now for a
>> couple of reasons:
>> - The existing Ant build (in addition to the code) has presumably been
>> broken by recent build related changes made in the main java tree on
>> trunk.
>> - When I release the initial qpid-parent pom in the coming hours it should
>> now all 'Just Work'* (using the remote snapshots unless you install a
>> local
>> version using the maven build in the main java tree).
>>
>> Robbie
>>
>> * Where for now 'Just Work' = compile, Rob etc will no doubt still need to
>> further check/fix the impact on actual functionality of changes they
>> have/are going to make. Doing this will make it easier for them to do
>> that.
>>
>> On 6 April 2014 18:09, Fraser Adams <fraser.adams@blueyonder.co.uk>
>> wrote:
>>
>>  Thanks again for all the responses Robbie.
>>>
>>> Go for it.
>>>
>>> If I get burned too badly I can always find out where you live
>>> <dr-evil>mwa mwa mwa mwa!!</dr-evil>
>>>
>>> Frase
>>>
>>>
>>>
>>> On 06/04/14 15:50, Robbie Gemmell wrote:
>>>
>>>  On 6 April 2014 10:29, Fraser Adams <fraser.adams@blueyonder.co.uk>
>>>> wrote:
>>>>
>>>>   Hi Robbie,
>>>>
>>>>> Firstly thanks so much for taking the time to give such a comprehensive
>>>>> response. I'll summarise my current thoughts below.
>>>>>
>>>>> 1. The directory structure in qpid/tools/src/java seems fine, I've got
>>>>> no
>>>>> strong aversion to it. There might be some tweakage, and possibly as
>>>>> you've
>>>>> suggested before there might be mileage in separating into console,
>>>>> agent,
>>>>> common+util as most applications using it would only tend to need the
>>>>> console, common and JMS stuff, but it's probably not worth doing too
>>>>> much
>>>>> there at the moment 'cause I've no idea what carnage I'll cause when
I
>>>>> start to (eventually) get round to doing AMQP 1.0 Management - quite
>>>>> how
>>>>> I
>>>>> handle any migration is very much TBD :-/
>>>>>
>>>>> 2. I can see how Maven can (sort of) simplify build scripts, though
>>>>> that
>>>>> only seems to apply to large projects like the Java Broker, for the QMF
>>>>> stuff, which is ultimately just a JMS client it seems to have gone from
>>>>> one
>>>>> fairly simple build.xml which was pretty straightforward to something
>>>>> like
>>>>> six pom.xml files - there's progress for you :-D
>>>>>
>>>>>
>>>>>   You only have to deal with the top level pom.xml generally when
>>>>> building
>>>>>
>>>> stuff, sure there may be more files 'behind' that but you basically dont
>>>> need to look at them unless you want to change them, at which point I
>>>> think
>>>> it actually becomes easier to find what you want to change because
>>>> things
>>>> are nicely split up into modules that deal with themselves, and a parent
>>>> that everything inherits.
>>>>
>>>> We could have left the non-broker-plugin stuff as one big aggregate jar,
>>>> but the reality is that the existing 'qmf.jar' contains a lot more than
>>>> a
>>>> single thing and so splitting it up to at least some extent seemed like
>>>> the
>>>> correct thing to do. The python/original QMF implementation and command
>>>> line tools built around it are separated in a similar way, presumably
>>>> for
>>>> similar reasons.
>>>>
>>>>
>>>>   3. In the qpid-qmf2-parent pom I don't really understand the
>>>>
>>>>> dependencies,
>>>>> that pom seems to be responsible for the top-level building of the
>>>>> modules,
>>>>> but why does it have dependencies of its own (log4j, slf4j, jms spec)?
>>>>> I
>>>>> would get it if this had the common dependencies of all the
>>>>> sub-modules,
>>>>> but the qpid-qmf2 pom has its own dependencies on slf4j and jms spec
-
>>>>> exposing more holes in my lack of understanding of Maven I guess.
>>>>>
>>>>>   First up, as with qpid-parent yesterday, qpid-qmf2-parent actually
>>>>> has
>>>>>
>>>> no
>>>> dependencies, but it does have a 'dependency management' section, which
>>>> is
>>>> used to ensure consistent alignment of dependency versions in the child
>>>> modules and make their poms slightly simpler. See
>>>> http://maven.apache.org/guides/introduction/introduction-to-dependency-
>>>> mechanism.html#Dependency_Management
>>>>
>>>> Next up, inheritance. The parent pom uses 'pom' packaging and doesnt
>>>> produce any output except itself, while the individual modules then use
>>>> it
>>>> as their parent and then inherit much of its contents - such as the
>>>> dependencyManagement section that in turn ensures they all use the same
>>>> versions for those things going forward.
>>>>
>>>> I could have left out the dependencyManagement section and just repeated
>>>> the SLF4J and JMS spec dependency etries in their entirity in every
>>>> module
>>>> requiring them, but that becomes harder to maintain going forward and is
>>>> more prone to becoming inconsistent.
>>>>
>>>> Some of this information is overriding bits from qpid-parent pom, which
>>>> currently contains a much larger amount of dependency management, but we
>>>> are considering moving some of that back to its immediate children as it
>>>> doesnt all apply universally to them. I also now realise that a (highly
>>>> unusual going forward) change I made yesterday to qpid-parent might not
>>>> sit
>>>> nice with the dependencyManagement section in qpid-qmf2-parent so I'll
>>>> go
>>>> fix that if required - which yes it was, update commmited :)
>>>>
>>>>
>>>>   4. The main benefit that I can see of Maven, aside from (sort of)
>>>>
>>>>> simplifying build scripts (which only seems to apply for large builds)
>>>>> is
>>>>> that it does seem to provide useful artefacts from the perspective of
>>>>> *users*. That is to say it's biggest (or only as far as I can see ;->)
>>>>> benefit is from the perspective of release packaging - you end up with
>>>>> some
>>>>> nice tar.gz files that contain what's needed, so I sort of get why it's
>>>>> nice from a user perspective - maybe that's worth the price of
>>>>> admission
>>>>> alone.
>>>>>
>>>>>   We always had the ability to make nice tar.gz files, its just much
>>>>> more
>>>>>
>>>> of
>>>> a pain in the ass to do so in our Ant build, often requiring significant
>>>> hoop jumping and typically coming out during the release process that
>>>> they
>>>> were wrong because we dont actually use them the rest of the time (due
>>>> to
>>>> the convenient but horrible global build dir that doesnt represent what
>>>> users actually get).
>>>>
>>>> For me the main benefits are the consistency (this obviously doesnt
>>>> apply
>>>> to you yet, but the many people who know maven from use on other
>>>> projects
>>>> will now largely be able to pick up the Qpid build and use most of it
>>>> without assistance, while the same wasn't really true of the Ant build
>>>> because so much of it is bespoke) and extending from that the ease of
>>>> integration with a lot of tooling that for the most part works simply
>>>> with
>>>> or is actually built into maven builds but usually requires pulling
>>>> teeth
>>>> (often mine) to get working with highly bespoke Ant builds such as ours.
>>>>
>>>> It will also mean we (i.e me) dont need to make a special effort to
>>>> publish
>>>> maven artifacts for components to maven central after our users request
>>>> it
>>>> (which they wont need to because we just will), as we (i.e me) always
>>>> have
>>>> in the past, or waste considerable amounts of [my] time verifying those
>>>> generated artifacts actually contain what they should every time we
>>>> release
>>>> (which they often dont - I have yet to check them this time round)
>>>> because
>>>> we dont actually use them the rest of the time.
>>>>
>>>>
>>>>   5. From a *dev* perspective though, at the moment the bit of the Maven
>>>>
>>>>> rework that a dislike the most is more or less exactly the same as
>>>>> point
>>>>> 4.
>>>>> above. In order to actually use the Java Broker I have to do mvn clean
>>>>> install,
>>>>>
>>>>>  You just need to run mvn package currently. You only need to use
>>>> install
>>>> if
>>>> you want to build something entirely separate using maven which declares
>>>> it
>>>> depends on the artifacts for the broker you just built. Strictly
>>>> speaking
>>>> you only need actually do that if you want to modify the broker first,
>>>> as
>>>> it will otherwise download the release or snapshot version (based on
>>>> your
>>>> specified dependency) from the remote repositories if you dont have it
>>>> in
>>>> your local repo already.
>>>>
>>>>
>>>>   I then need to take the tar.gz from <qpid>/java/broke/target copy
it
>>>>
>>>>> somewhere and unzip it - inconvenient!
>>>>>
>>>>>  As mentioned yesterday, its simply not finished. Removing this need
>>>> is a
>>>> 'nice to have' thing in my mind, which has ranked it below some other
>>>> 'needs to be done before this is correct / suitable for use at all'
>>>> things.
>>>>
>>>> I have basically viewed this personally along the lines of....until we
>>>> do
>>>> put something into the poms to start or unpack the broker, we actually
>>>> have
>>>> to use/do something a user e.g. looking at the website would do? Oh the
>>>> horror ;)
>>>>
>>>>
>>>>   But that's not the worse bit, when I need to figure out what's broken
>>>>
>>>>> next
>>>>> time the plugin API changes there's a whole world of copying and
>>>>> unzipping
>>>>> to be had, which as well as being inconvenient is prone to finger
>>>>> trouble.
>>>>>
>>>>>  Build and install the broker artifacts (or dont and it will just use
>>>> the
>>>> latest snapshot). Build the QMF bits, which will fail if we broke
>>>> something. Dont touch the tar file, or any jar files, or set the
>>>> classpath.
>>>> This stage is far easier than it was before with the Ant build for the
>>>> QMF
>>>> bits.
>>>>
>>>> When it comes to running it all up to make sure it works, yes you will
>>>> need
>>>> to copy some things *currently*. That could certainly be done via config
>>>> in
>>>> the poms if needed to 'automate' it. Its probably worth noting that
>>>> having
>>>> some/any real automated tests for these bits would mean that this step
>>>> wouldnt even be necessary much of the time.
>>>>
>>>> A key thing I think is being overlooked is that we will simply try
>>>> harder
>>>> not to break it in future. Because it wont be a complete pain to build
>>>> as
>>>> it is now, we will far more easily be able to ensure it is kept up to
>>>> date.
>>>> We would also now be able to very easily stick it in CI to make sure
>>>> that
>>>> we know we broke it if we hadnt noticed in advance (which I expect we
>>>> will
>>>> more, because it wont be an ordeal to do so anymore).
>>>>
>>>>
>>>>   It's unfortunate that Maven install only "installs" things to the
>>>>
>>>>> repository and that there doesn't seem to be a way to install assembly
>>>>> artefacts onto my system. I guess I could write my own private script
>>>>> that
>>>>> does all that - but it is as I say inconvenient from a dev perspective.
>>>>>
>>>>>   Thats the primary purpose of mvn install. We can however bind
>>>>> anything
>>>>>
>>>> we
>>>> like to any of the build phases, perhaps adding a profile that unpacks
>>>> to
>>>> a
>>>> specified directory as a 'system installation' type command etc etc. As
>>>> I
>>>> said yesterday, there are a multitude of options here, it just isnt
>>>> something we have got to *yet* because it really is only a convenience
>>>> whereas other things are not.
>>>>
>>>>
>>>>   6. Hmmm so I say item 5 was what I dislike the most, actually one
>>>> thing I
>>>>
>>>>> dislike just a bit more is what it does for WebApp development. What
I
>>>>> really like about programming in JavaScript is that I can just make a
>>>>> change, hit refresh and bingo, none of this build/compile malarkey. Now
>>>>> from a user perspective it's great that the QMF WebApp is neatly
>>>>> packaged
>>>>> in qpid-qmf2-tools.tar.gz but from a dev perspective it's a car crash
>>>>> waiting to happen :-( In order to run QpidRestAPI I have to unpack the
>>>>> tools, but that will then use the packaged WebApp by default, now there
>>>>> is
>>>>> a command line option so I can point webroot to
>>>>> <qpid>/tools/src/java/qpid-qmf2-tools/bin/qpid-web
>>>>> on trunk, so it's not like I can't work around it, but how long do you
>>>>> give
>>>>> me before I start accidentally making changes in the "installed"
>>>>> version
>>>>> -
>>>>> then accidentally lose them all next time I update the QpidRestAPI. I
>>>>> might
>>>>> be lucky, but I have a very simple brain :-)
>>>>>
>>>>>   You could probably make a profile to it run the GUI straight out of
>>>>> the
>>>>>
>>>> src
>>>> tree without ever building the package. Thats basically how the 'test'
>>>> classes work for example since it doesnt even need to build the jar
>>>> files
>>>> to run them.
>>>>
>>>>
>>>>   7. I'm a bit worried about the Maven version stuff. I can see it
>>>> being a
>>>>
>>>>> good thing wrt. being able to do reproducible builds, but does it
>>>>> really
>>>>> mean that every time a new Qpid release comes out there will be a need
>>>>> to
>>>>> try and remember to go through dozens of pom.xml files to update
>>>>> various
>>>>> artefact versions? Given that there are six poms for the QMF stuff
>>>>> alone
>>>>> that doesn't feel brilliant.
>>>>>
>>>>>   There are plugins that can help with this.
>>>>>
>>>>
>>>>   8. It feels like I'm pushed into Mavenising every other little Qpid
>>>> test
>>>>
>>>>> app I've got on my system. Before I had things (albeit perhaps messily)
>>>>> on
>>>>> my CLASSPATH, but now not.
>>>>>
>>>>>  As you say, this new way is cleaner, but you cancertainly do basically
>>>> what
>>>> you did before if you like, you will just have to specify more locations
>>>> in
>>>> your classpath.
>>>>
>>>> For production things that's fine and the Maven packaging does seem
>>>> quite
>>>>
>>>>  good, but for all the other hacky little things .... sigh, I really do
>>>>> think Maven is a virus, once one thing catches it you have to "update"
>>>>> everything else.
>>>>>
>>>>>   You arent really forced in any way in my mind. Sure, it makes
>>>>> something
>>>>>
>>>> you
>>>> are doing slightly uglier, but you are entirely free to keep doing that
>>>> hacky thing if you like.
>>>>
>>>> I feel a related consideration also needs to be applied in the opposite
>>>> direction, i.e the folks (lots of them) that have long had maven builds
>>>> and
>>>> might come along and want to use the output of our Ant build. Poor sods.
>>>>
>>>>
>>>>   9. All that said, there seems to be an inexorable trend towards
>>>> Maven, so
>>>>
>>>>> from my perspective you might as well move forward on the great Qpid
>>>>> Mavenisation. I'm not going to stand in the way even if I might moan
a
>>>>> bit
>>>>> until I get used to it :-D.
>>>>>
>>>>>  Feel free to moan away :)
>>>>
>>>> It'd be nice though if there were some solutions to the productivity
>>>> issues
>>>>
>>>>
>>>> I realise that this transition might involve a temporary productivity
>>>> dip
>>>> for some who arent as used to it already, but also consider the
>>>> productivity increase others will get from it (I'll be honest: whilst I
>>>> think that does apply generally, I do primarily mean me, who wont have
>>>> to
>>>> spend so much time rewriting chunks the Ant build to do really simple
>>>> things, or waste as much time verifying the output we rarely generate is
>>>> actually correct, or <insert time sinks here>..) and that a key driver
>>>> is
>>>> allowing us to progress towards making the project as a whole more
>>>> component focused, which will itself improve productivity and wouldnt be
>>>> possible without rewriting what we had anyway. We will get to the little
>>>> things like starting the broker one its built.
>>>>
>>>> There is obviously a learning curve, but its not that big, once you are
>>>> over it I actually think you would actually come to see increased
>>>> productivity from using it generally instead of writing Ant scripts (or
>>>> dealing directly with javac) and tinkering with the CLASSPATH for
>>>> example.
>>>>
>>>>
>>>>   - I already struggle with productivity given limited coding time
>>>> between
>>>>
>>>>> work and family stuff, some weekends I feel like I've wasted it all
>>>>> chasing
>>>>> my tail rather than doing anything useful.
>>>>>
>>>>> Regards,
>>>>> Frase
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 05/04/14 18:57, Robbie Gemmell wrote:
>>>>>
>>>>>   On 4 April 2014 14:41, Fraser Adams <fraser.adams@blueyonder.co.uk>
>>>>>
>>>>>> wrote:
>>>>>>
>>>>>>    On 04/04/14 13:30, Rob Godfrey wrote:
>>>>>>
>>>>>>     No... "install in the local repo" is what the mvn clean install
>>>>>>> will
>>>>>>>
>>>>>>>  do... So, you literally just have to follow Robbie's instructions:
>>>>>>>> check
>>>>>>>> out the qpid-parent-pom/trunk directory, run mvn clean install
from
>>>>>>>> wherever you checked it out to... then go back to the broker
and the
>>>>>>>> maven
>>>>>>>> build should complete ok. Hope this helps, Rob
>>>>>>>>
>>>>>>>>    Cheers Rob,
>>>>>>>>
>>>>>>>>  I ended up figuring out that myself after staring at the
>>>>>>> instructions
>>>>>>> for
>>>>>>> a while and deciding that I was reading way too much into things
:-)
>>>>>>>
>>>>>>> I've now got the Java Broker etc. built using Maven but before
I play
>>>>>>> with
>>>>>>> the QMF things I wouldn't mind answers to a few Maven questions
- I'm
>>>>>>> still
>>>>>>> very much at the "Burn the Witch" stage wrt. my trust of Maven
;-)
>>>>>>>
>>>>>>> I've now ended up with a directory
>>>>>>> /home/fadams/.m2/repository
>>>>>>>
>>>>>>> Filled with stuff.
>>>>>>>
>>>>>>> I'm not clear by what witchcraft Maven decides what to shove
there -
>>>>>>>
>>>>>>>   Maven has the concept of a local repo and a remote repo, analogous
>>>>>>> to
>>>>>>>
>>>>>> e.g.
>>>>>> your installed packages in a linux distro, and the repository the
>>>>>> package
>>>>>> manager got them from. The default local repo localtion is
>>>>>> ~/.m2/repository, and the default remote repo is maven central.
>>>>>>
>>>>>> Any dependencies it needs to download will go in the local repo,
so it
>>>>>> doenst necessarily need to download them again next time they are
used
>>>>>> (unles e.g they are snapshots and new versions came out).
>>>>>> Additionally,
>>>>>> anything you run "mvn install" on will go in the local repo configured
>>>>>> at
>>>>>> the time (can be overriden per-command for example) to allow it to
be
>>>>>> used
>>>>>> by other things you build later.
>>>>>>
>>>>>> 'Any dependencies' above includes the dependencies of a particular
>>>>>> artifact
>>>>>> you are using, the [plugin] dependencies of anything needed by your
>>>>>> build
>>>>>> process to do what you are asking, and any [plugin] dependencies
>>>>>> needed
>>>>>> by
>>>>>> maven itself.
>>>>>>
>>>>>> (Additional reading, there are different dependency scopes that can
>>>>>> influence if/when they are needed/used etc:
>>>>>> http://maven.apache.org/guides/introduction/
>>>>>> introduction-to-dependency-
>>>>>> mechanism.html#Dependency_Scope
>>>>>> )
>>>>>>
>>>>>>
>>>>>>    for example I eyeballed the parent pom.xml and saw the
>>>>>> dependencies,
>>>>>> The parent pom actually has no real dependencies (except on its
>>>>>> parent,
>>>>>> the
>>>>>> apache pom).
>>>>>>
>>>>>> What you likely saw was the pluginManagement and dependencyManagement
>>>>>> sections, which help fix down and/or consistently specify the versions
>>>>>> of
>>>>>> particular dependencies and plugins that might be used by the child
>>>>>> modules
>>>>>> and their build steps. These dont actually mean things have a
>>>>>> dependency
>>>>>> on
>>>>>> those artifacts, just which version etc they would get by default
if
>>>>>> they
>>>>>> should happen to specify such a dependency or plugin to use. This
>>>>>> helps
>>>>>> keep things using consistent versions of artifacts, and in the case
of
>>>>>> plugins helps ensure reproducable builds by locking down the versions
>>>>>> in
>>>>>> use.
>>>>>>
>>>>>>
>>>>>>    but when I looked at the repository after running mvn clean
>>>>>> install on
>>>>>>
>>>>>>  that I was kind of expecting the directories in the repository to
>>>>>>> pretty
>>>>>>> much match what was in the dependencies, but it definitely didn't
>>>>>>> after
>>>>>>> running mvn on the parent pom.xml, though to be fair after building
>>>>>>> the
>>>>>>> main things it does all appear to match (though with a ton of
other
>>>>>>> stuff
>>>>>>> too).
>>>>>>>
>>>>>>>    What you would have got after doing 'mvn install' on the
>>>>>>> qpid-parent
>>>>>>> pom
>>>>>>>
>>>>>>>  would be mostly only the plugins needed by maven itself to carry
out
>>>>>> those
>>>>>> actions. What they ship in the core maven release is augmented by
>>>>>> downloading/installing things to the local repo as they are used
both
>>>>>> by
>>>>>> itself and by people declaring they want to use a specific plugin
in
>>>>>> their
>>>>>> build.
>>>>>>
>>>>>>
>>>>>>    When I build the Java Broker etc. it seemed to take an age, when
I
>>>>>> was
>>>>>>
>>>>>>  using ant it took under a minute on my system, but with Maven it
>>>>>>> reports
>>>>>>> a
>>>>>>> total time of 5:18 min and it looks like it was downloading half
the
>>>>>>> Internet :-D I'm *guessing* that this is a one-off cost as it
fills
>>>>>>> up
>>>>>>> my
>>>>>>> local repo with stuff?
>>>>>>>
>>>>>>>
>>>>>>>    Yes, the first time you would have incurred the cost of grabbing
>>>>>>> any
>>>>>>>
>>>>>>>  plugins the build needed that you didnt already have and also
the
>>>>>> dependencies of the things being built (the later of which happens
on
>>>>>> your
>>>>>> first Ant build too - go look in ~/.ivy2/cache, then try deleting
it
>>>>>> and
>>>>>> see what happens to the Ant build time). Run it again and it will
be a
>>>>>> lot
>>>>>> faster going forward.
>>>>>>
>>>>>> On my machine using the head of trunk (I just committed a bunch of
>>>>>> tweaks
>>>>>> earlier), 'ant clean build' vs 'mvn clean package -DskipTests=true'
>>>>>> has
>>>>>> the
>>>>>> maven build being 1 or 2sec slower (32 vs 33/34), which is actually
>>>>>> surprisingly close since using those commands we currenty have the
>>>>>> maven
>>>>>> build doing a bunch of extra packaging and enforcement that the ant
>>>>>> build
>>>>>> doesnt. It may also be quicker if told to build 'offline' to ensure
it
>>>>>> didnt check any repos for updates.
>>>>>>
>>>>>> (It takes a further 2mins 21sec on top of that for me to use the
Ant
>>>>>> build
>>>>>> to generate the maven artifacts we currently publish to central...a
>>>>>> generation step we obviously wont need to do for the maven build
as
>>>>>> they
>>>>>> are already there inherantly.)
>>>>>>
>>>>>>
>>>>>>    Where do you set CLASSPATH when you build using Maven? With the
ant
>>>>>> build
>>>>>>
>>>>>>  I used to have:
>>>>>>> <qpid>qpid/java/build/lib/qpid-all.jar
>>>>>>>
>>>>>>>
>>>>>>>    You dont set CLASSPATH, basically. Supplying dependency
>>>>>>> information
>>>>>>> is
>>>>>>>
>>>>>>>  all
>>>>>> handled through the poms, which maven uses to build the classpath
for
>>>>>> each
>>>>>> module as it builds. If you want to build something else that uses
e.g
>>>>>> the
>>>>>> qpid client, the thing your are building should have a dependency
for
>>>>>> qpid-client in its pom. You can make that dependency be either a
>>>>>> published
>>>>>> release [or snapshot] version available in central [or the apache
>>>>>> snapshots
>>>>>> repo], or you can 'mvn install' your own modified copy if you want
to
>>>>>> use
>>>>>> an unpublished client release. If you dont want to use maven / want
to
>>>>>> use
>>>>>> basic javac commands for the other thing you are compiling against
the
>>>>>> client, then you would need to make the client jars available
>>>>>> somewhere
>>>>>> and
>>>>>> add the individual jars to your classpath currently.
>>>>>>
>>>>>> We aren't recreating qpid-all at this time because while it may be
>>>>>> convenient in cases, it is actually a horrible horrible thing, which
>>>>>> I'll
>>>>>> talk a bit more to later when covering the build directory :)
>>>>>>
>>>>>> We have a qpid-all file in every release artifact made by the ant
>>>>>> build,
>>>>>> each having different content, and all different from the one that
>>>>>> gets
>>>>>> created in the build dir. Even if we wanted to recreate it in the
>>>>>> maven
>>>>>> build, we would need to give it a unique name for every component
to
>>>>>> ever
>>>>>> publish it anyway.
>>>>>>
>>>>>> We could instead do something like add the classpath manifest entries
>>>>>> previously contained in qpid-all.jar into the main qpid-client,
>>>>>> qpid-broker
>>>>>> etc jars so you could do 'java -jar qpid-foo' etc on them instead.
>>>>>>
>>>>>>
>>>>>>    Which was nice and convenient, if possibly a bit sloppy, from
what
>>>>>> I
>>>>>> can
>>>>>>
>>>>>>  see each qpid/java subdirectory seems to have a target directory
e.g.
>>>>>>> qpid/java/client/target though in there there is
>>>>>>> qpid-client-0.28-SNAPSHOT.jar which seems less convenient that
>>>>>>> qpid-client.jar if I want to set up CLASSPATH in my .bashrc.
Am I
>>>>>>> missing
>>>>>>> something?
>>>>>>>
>>>>>>> Another thing that I'm not clear on is that in the Maven repository
>>>>>>> there
>>>>>>> seems to be a bunch of jars installed - as an example
>>>>>>> org/eclipse/jetty/jetty-websocket/8.1.14.v20131031/
>>>>>>> jetty-websocket-8.1.14.v20131031.jar.
>>>>>>> In the "olden days" Qpid pulled in jetty and the Jar was available
in
>>>>>>> qpid/java/build/lib so when I was messing around with Jetty on
>>>>>>> another
>>>>>>> project I had it handily on my CLASSPATH - I can't seem to see
jetty
>>>>>>> anywhere now except in the Maven repository? So how does the
Broker
>>>>>>> see
>>>>>>> it?
>>>>>>>
>>>>>>>    The broker sees it because it says it has a runtime dependency
on
>>>>>>> the
>>>>>>>
>>>>>>>  management-http module in its pom, and the management-http module
>>>>>> says
>>>>>> it
>>>>>> has a compile time dependency on Jetty in its pom, and so maven then
>>>>>> knows
>>>>>> the broker transitively needs jetty. The actual jars never leave
the
>>>>>> local
>>>>>> repo unless we instruct the build to do something that requires that
>>>>>> (e.g.
>>>>>> package a binary assembly containing it, or copying it somewhere)
in
>>>>>> the
>>>>>> pom or you execute a command manually that does (e.g. a quick Google
>>>>>> says
>>>>>> 'mvn dependency:copy-dependencies' requests the dependency plugin
to
>>>>>> run
>>>>>> its copy-dependencies goal, which create a dir in target/dependency
>>>>>> with
>>>>>> everything in it).
>>>>>>
>>>>>> The only reason the Ant build still puts things in the lib dir (by
>>>>>> using
>>>>>> Ivy to grab thigns from the maven central repo, i.e roughly the same
>>>>>> thing
>>>>>> Maven is doing to download things) is so that I didnt need to rewrite
>>>>>> even
>>>>>> more of the Ant build than I already was when I modified it to allow
>>>>>> removing the jars from the lib dir, where they were actually committed
>>>>>> into
>>>>>> the svn repo historically (I refer you to my previous mails comment
>>>>>> about
>>>>>> the Ant build being a pain in the ass and me being the one best placed
>>>>>> to
>>>>>> speak best to this point :P).
>>>>>>
>>>>>>
>>>>>>
>>>>>>    In the olden days when I just did "ant" at the end of that I ended
>>>>>> up
>>>>>>
>>>>>>  with
>>>>>>> something that would run 'cause I had
>>>>>>> <qpid-trunk>qpid/java/build/bin
>>>>>>>
>>>>>>> on my path and could simply do "qpid-server" but now there's
no nice
>>>>>>> convenient build directory.
>>>>>>>
>>>>>>>    The existing build directory is horrible and should burn :P
>>>>>>>
>>>>>>>  I'll admit, it can be convenient, but its also a complete jumble
>>>>>> with a
>>>>>> little bit of extra mess on top, mixing up all the different
>>>>>> components
>>>>>> together in lib so you cant tell which bits need what (qpid-all.jar
in
>>>>>> the
>>>>>> build directory being worst of all, as you have everything on the
>>>>>> classpath
>>>>>> manifest in there) and doing ugly copies of stuff into subdirs of
lib
>>>>>> to
>>>>>> seperate them for later use precisely because you cant tell which
bits
>>>>>> need
>>>>>> what. I would have tried to kill it a long time ago if it didnt
>>>>>> involve
>>>>>> yet
>>>>>> more 'rewriting the ant build' fun, which I just couldnt bring myself
>>>>>> to
>>>>>> subject me to :)
>>>>>>
>>>>>> Maven deals with modules, which each have their own target dir for
>>>>>> their
>>>>>> intermediate work and final output, and if you want something that
>>>>>> aggregates multiple modules then there are varous ways to do that,
>>>>>> avoiding
>>>>>> the global build dir mess while still letting you compose a larger
>>>>>> unit
>>>>>> from the individual bits. It is certainly different than what our
Ant
>>>>>> build
>>>>>> does, but in this case I think its a good thing. We can always do
>>>>>> things
>>>>>> to
>>>>>> make life easier, more below.
>>>>>>
>>>>>>
>>>>>>    I noticed in
>>>>>>
>>>>>>  qpid/java/broker/target/qpid-broker-0.28-SNAPSHOT-bin.tar.gz that
>>>>>>> archive
>>>>>>> seems to contain qpid-server, do I *really* have to now have
to faff
>>>>>>> around
>>>>>>> copying and unpacking archives everytime I want to update a build?
>>>>>>> That
>>>>>>> seems an awful lot less convenient than simply doing "ant clean"
then
>>>>>>> "ant".
>>>>>>>
>>>>>>> Have I missed something? I hope so 'cause if not it seems a step
>>>>>>> backwards
>>>>>>> from a "just works" POV
>>>>>>>
>>>>>>>    What you say is true *currently*, to run the broker built
by the
>>>>>>> maven
>>>>>>>
>>>>>>>  build you would need to unpack that tar file. We can certainly
add
>>>>>> functionality to remove that need, we just havent got to those final
>>>>>> few
>>>>>> 'nice to have' things yet. There are a multitude of ways we could
do
>>>>>> it,
>>>>>> for example we could use a plugin and have the maven build itself
>>>>>> start
>>>>>> the
>>>>>> broker (e.g see what i did with the 'tests' in the QMF tree: those
>>>>>> actually
>>>>>> run without even creating a jar file), or add some build config to
>>>>>> allow
>>>>>> output of an unpackaged version of it during the main build,
>>>>>> or..<insert
>>>>>> idea>.
>>>>>>
>>>>>>
>>>>>>    This might all be second nature to folks familiar with Maven,
but
>>>>>> I'm
>>>>>> a
>>>>>>
>>>>>>  bit "old skool" and I quite like knowing what's installed on my
>>>>>>> system,
>>>>>>> where it is installed, and why it's there so I'd quite like a
bit of
>>>>>>> reassurance :-)
>>>>>>>
>>>>>>> Frase
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ------------------------------------------------------------
>>>>>>> ---------
>>>>>>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>>>>>>> For additional commands, e-mail: users-help@qpid.apache.org
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>   ------------------------------------------------------------
>>>>>>> ---------
>>>>>>>
>>>>>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>>>>> For additional commands, e-mail: users-help@qpid.apache.org
>>>>>
>>>>>
>>>>>
>>>>>  ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>>> For additional commands, e-mail: users-help@qpid.apache.org
>>>
>>>
>>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message