qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robbie Gemmell <robbie.gemm...@gmail.com>
Subject Re: Maven build for the QMF2 tools
Date Sat, 05 Apr 2014 18:06:26 GMT
On 5 April 2014 18:57, Robbie Gemmell <robbie.gemmell@gmail.com> wrote:

>
>
>
> On 4 April 2014 14:41, Fraser Adams <fraser.adams@blueyonder.co.uk> wrote:
>
>> On 04/04/14 13:30, Rob Godfrey wrote:
>>
>>> No... "install in the local repo" is what the mvn clean install will
>>> do... So, you literally just have to follow Robbie's instructions: check
>>> out the qpid-parent-pom/trunk directory, run mvn clean install from
>>> wherever you checked it out to... then go back to the broker and the maven
>>> build should complete ok. Hope this helps, Rob
>>>
>> Cheers Rob,
>> I ended up figuring out that myself after staring at the instructions for
>> a while and deciding that I was reading way too much into things :-)
>>
>> I've now got the Java Broker etc. built using Maven but before I play
>> with the QMF things I wouldn't mind answers to a few Maven questions - I'm
>> still very much at the "Burn the Witch" stage wrt. my trust of Maven ;-)
>>
>> I've now ended up with a directory
>> /home/fadams/.m2/repository
>>
>> Filled with stuff.
>>
>> I'm not clear by what witchcraft Maven decides what to shove there -
>
>
> Maven has the concept of a local repo and a remote repo, analogous to e.g.
> your installed packages in a linux distro, and the repository the package
> manager got them from. The default local repo localtion is
> ~/.m2/repository, and the default remote repo is maven central.
>
> Any dependencies it needs to download will go in the local repo, so it
> doenst necessarily need to download them again next time they are used
> (unles e.g they are snapshots and new versions came out). Additionally,
> anything you run "mvn install" on will go in the local repo configured at
> the time (can be overriden per-command for example) to allow it to be used
> by other things you build later.
>
> 'Any dependencies' above includes the dependencies of a particular
> artifact you are using, the [plugin] dependencies of anything needed by
> your build process to do what you are asking, and any [plugin] dependencies
> needed by maven itself.
>
> (Additional reading, there are different dependency scopes that can
> influence if/when they are needed/used etc:
> http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Scope
> )
>
>
>> for example I eyeballed the parent pom.xml and saw the dependencies,
>
>
> The parent pom actually has no real dependencies (except on its parent,
> the apache pom).
>
> What you likely saw was the pluginManagement and dependencyManagement
> sections, which help fix down and/or consistently specify the versions of
> particular dependencies and plugins that might be used by the child modules
> and their build steps. These dont actually mean things have a dependency on
> those artifacts, just which version etc they would get by default if they
> should happen to specify such a dependency or plugin to use. This helps
> keep things using consistent versions of artifacts, and in the case of
> plugins helps ensure reproducable builds by locking down the versions in
> use.
>
>
>> but when I looked at the repository after running mvn clean install on
>> that I was kind of expecting the directories in the repository to pretty
>> much match what was in the dependencies, but it definitely didn't after
>> running mvn on the parent pom.xml, though to be fair after building the
>> main things it does all appear to match (though with a ton of other stuff
>> too).
>>
>
> What you would have got after doing 'mvn install' on the qpid-parent pom
> would be mostly only the plugins needed by maven itself to carry out those
> actions. What they ship in the core maven release is augmented by
> downloading/installing things to the local repo as they are used both by
> itself and by people declaring they want to use a specific plugin in their
> build.
>
>
>>
>> When I build the Java Broker etc. it seemed to take an age, when I was
>> using ant it took under a minute on my system, but with Maven it reports a
>> total time of 5:18 min and it looks like it was downloading half the
>> Internet :-D I'm *guessing* that this is a one-off cost as it fills up my
>> local repo with stuff?
>>
>>
> Yes, the first time you would have incurred the cost of grabbing any
> plugins the build needed that you didnt already have and also the
> dependencies of the things being built (the later of which happens on your
> first Ant build too - go look in ~/.ivy2/cache, then try deleting it and
> see what happens to the Ant build time). Run it again and it will be a lot
> faster going forward.
>
> On my machine using the head of trunk (I just committed a bunch of tweaks
> earlier), 'ant clean build' vs 'mvn clean package -DskipTests=true' has the
> maven build being 1 or 2sec slower (32 vs 33/34), which is actually
> surprisingly close since using those commands we currenty have the maven
> build doing a bunch of extra packaging and enforcement that the ant build
> doesnt. It may also be quicker if told to build 'offline' to ensure it
> didnt check any repos for updates.
>

I should have said 'ant clean build -Doptional=true' there, since thats the
comparable module build set and what I typically use / was actually running
earlier :)

>
> (It takes a further 2mins 21sec on top of that for me to use the Ant build
> to generate the maven artifacts we currently publish to central...a
> generation step we obviously wont need to do for the maven build as they
> are already there inherantly.)
>
>
>> Where do you set CLASSPATH when you build using Maven? With the ant build
>> I used to have:
>> <qpid>qpid/java/build/lib/qpid-all.jar
>>
>>
> You dont set CLASSPATH, basically. Supplying dependency information is all
> handled through the poms, which maven uses to build the classpath for each
> module as it builds. If you want to build something else that uses e.g the
> qpid client, the thing your are building should have a dependency for
> qpid-client in its pom. You can make that dependency be either a published
> release [or snapshot] version available in central [or the apache snapshots
> repo], or you can 'mvn install' your own modified copy if you want to use
> an unpublished client release. If you dont want to use maven / want to use
> basic javac commands for the other thing you are compiling against the
> client, then you would need to make the client jars available somewhere and
> add the individual jars to your classpath currently.
>
> We aren't recreating qpid-all at this time because while it may be
> convenient in cases, it is actually a horrible horrible thing, which I'll
> talk a bit more to later when covering the build directory :)
>
> We have a qpid-all file in every release artifact made by the ant build,
> each having different content, and all different from the one that gets
> created in the build dir. Even if we wanted to recreate it in the maven
> build, we would need to give it a unique name for every component to ever
> publish it anyway.
>
> We could instead do something like add the classpath manifest entries
> previously contained in qpid-all.jar into the main qpid-client, qpid-broker
> etc jars so you could do 'java -jar qpid-foo' etc on them instead.
>
>
>> Which was nice and convenient, if possibly a bit sloppy, from what I can
>> see each qpid/java subdirectory seems to have a target directory e.g.
>> qpid/java/client/target though in there there is
>> qpid-client-0.28-SNAPSHOT.jar which seems less convenient that
>> qpid-client.jar if I want to set up CLASSPATH in my .bashrc. Am I missing
>> something?
>>
>> Another thing that I'm not clear on is that in the Maven repository there
>> seems to be a bunch of jars installed - as an example
>> org/eclipse/jetty/jetty-websocket/8.1.14.v20131031/
>> jetty-websocket-8.1.14.v20131031.jar. In the "olden days" Qpid pulled in
>> jetty and the Jar was available in qpid/java/build/lib so when I was
>> messing around with Jetty on another project I had it handily on my
>> CLASSPATH - I can't seem to see jetty anywhere now except in the Maven
>> repository? So how does the Broker see it?
>>
>
> The broker sees it because it says it has a runtime dependency on the
> management-http module in its pom, and the management-http module says it
> has a compile time dependency on Jetty in its pom, and so maven then knows
> the broker transitively needs jetty. The actual jars never leave the local
> repo unless we instruct the build to do something that requires that (e.g.
> package a binary assembly containing it, or copying it somewhere) in the
> pom or you execute a command manually that does (e.g. a quick Google says
> 'mvn dependency:copy-dependencies' requests the dependency plugin to run
> its copy-dependencies goal, which create a dir in target/dependency with
> everything in it).
>
> The only reason the Ant build still puts things in the lib dir (by using
> Ivy to grab thigns from the maven central repo, i.e roughly the same thing
> Maven is doing to download things) is so that I didnt need to rewrite even
> more of the Ant build than I already was when I modified it to allow
> removing the jars from the lib dir, where they were actually committed into
> the svn repo historically (I refer you to my previous mails comment about
> the Ant build being a pain in the ass and me being the one best placed to
> speak best to this point :P).
>
>
>
>> In the olden days when I just did "ant" at the end of that I ended up
>> with something that would run 'cause I had
>> <qpid-trunk>qpid/java/build/bin
>>
>> on my path and could simply do "qpid-server" but now there's no nice
>> convenient build directory.
>>
>
> The existing build directory is horrible and should burn :P
>
> I'll admit, it can be convenient, but its also a complete jumble with a
> little bit of extra mess on top, mixing up all the different components
> together in lib so you cant tell which bits need what (qpid-all.jar in the
> build directory being worst of all, as you have everything on the classpath
> manifest in there) and doing ugly copies of stuff into subdirs of lib to
> seperate them for later use precisely because you cant tell which bits need
> what. I would have tried to kill it a long time ago if it didnt involve yet
> more 'rewriting the ant build' fun, which I just couldnt bring myself to
> subject me to :)
>
> Maven deals with modules, which each have their own target dir for their
> intermediate work and final output, and if you want something that
> aggregates multiple modules then there are varous ways to do that, avoiding
> the global build dir mess while still letting you compose a larger unit
> from the individual bits. It is certainly different than what our Ant build
> does, but in this case I think its a good thing. We can always do things to
> make life easier, more below.
>
>
>> I noticed in
>> qpid/java/broker/target/qpid-broker-0.28-SNAPSHOT-bin.tar.gz that
>> archive seems to contain qpid-server, do I *really* have to now have to
>> faff around copying and unpacking archives everytime I want to update a
>> build? That seems an awful lot less convenient than simply doing "ant
>> clean" then "ant".
>>
>> Have I missed something? I hope so 'cause if not it seems a step
>> backwards from a "just works" POV
>>
>
> What you say is true *currently*, to run the broker built by the maven
> build you would need to unpack that tar file. We can certainly add
> functionality to remove that need, we just havent got to those final few
> 'nice to have' things yet. There are a multitude of ways we could do it,
> for example we could use a plugin and have the maven build itself start the
> broker (e.g see what i did with the 'tests' in the QMF tree: those actually
> run without even creating a jar file), or add some build config to allow
> output of an unpackaged version of it during the main build, or..<insert
> idea>.
>
>
>>
>> This might all be second nature to folks familiar with Maven, but I'm a
>> bit "old skool" and I quite like knowing what's installed on my system,
>> where it is installed, and why it's there so I'd quite like a bit of
>> reassurance :-)
>>
>> Frase
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>> For additional commands, e-mail: users-help@qpid.apache.org
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message