Return-Path: X-Original-To: apmail-qpid-users-archive@www.apache.org Delivered-To: apmail-qpid-users-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id C972E1068E for ; Fri, 2 May 2014 11:28:40 +0000 (UTC) Received: (qmail 84424 invoked by uid 500); 2 May 2014 11:28:40 -0000 Delivered-To: apmail-qpid-users-archive@qpid.apache.org Received: (qmail 84035 invoked by uid 500); 2 May 2014 11:28:39 -0000 Mailing-List: contact users-help@qpid.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: users@qpid.apache.org Delivered-To: mailing list users@qpid.apache.org Received: (qmail 84025 invoked by uid 99); 2 May 2014 11:28:38 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 02 May 2014 11:28:38 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of robbie.gemmell@gmail.com designates 209.85.220.171 as permitted sender) Received: from [209.85.220.171] (HELO mail-vc0-f171.google.com) (209.85.220.171) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 02 May 2014 11:28:34 +0000 Received: by mail-vc0-f171.google.com with SMTP id lc6so404365vcb.2 for ; Fri, 02 May 2014 04:28:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=nGt9jqwq216KUxcMkCckb8uYGZF74rOKKko7LeNgkUw=; b=vxvOGAQfoFD1lLbfcYrUEJVg3Fpt76SoVrArE9m1XS4JXzC8LT5LRHcyNzTR7ib0uX xUVgKUkuwcIYlyGGE+SVVZwkGxNZp0Gd+phwqfVxQwvCKAES5xqiwHix9mrE4Ss0jt3I dJx2c0Nu6VFuIv8Kreq0b4X3XgAkYOmlxOoyAIdbWqb6K7BZ7hR4mad6yviCG7UdFNb3 +qWzaDR6H7T4gUrLFKPfdQBazvtR6gR5mpce9JBiRdW81YUrqumKEbeZqin/s0aSEkFZ B69+Aast8DrVHCpdI4eHDYijy+r++++s1wlLA0fENkDIf9fKjKSfyLQpiDbuFvAUU+Uy Y2xA== MIME-Version: 1.0 X-Received: by 10.58.13.104 with SMTP id g8mr13172333vec.16.1399030093248; Fri, 02 May 2014 04:28:13 -0700 (PDT) Received: by 10.58.38.225 with HTTP; Fri, 2 May 2014 04:28:13 -0700 (PDT) In-Reply-To: <53637237.6030903@blueyonder.co.uk> References: <531F5FA4.1030700@blueyonder.co.uk> <5321F98A.40001@blueyonder.co.uk> <533E8FC1.30502@blueyonder.co.uk> <533EB66F.2090301@blueyonder.co.uk> <53411E7B.90905@blueyonder.co.uk> <53418A65.2050801@blueyonder.co.uk> <53637237.6030903@blueyonder.co.uk> Date: Fri, 2 May 2014 12:28:13 +0100 Message-ID: Subject: Re: Maven build for the QMF2 tools From: Robbie Gemmell To: "users@qpid.apache.org" Content-Type: multipart/alternative; boundary=047d7b2e7386bcfe6d04f8691378 X-Virus-Checked: Checked by ClamAV on apache.org --047d7b2e7386bcfe6d04f8691378 Content-Type: text/plain; charset=UTF-8 On 2 May 2014 11:23, Fraser Adams wrote: > I'd be content as long as the " > > > When I release the initial qpid-parent pom in the coming hours it should > now all 'Just Work'* > > " > caveat is in place :-) > Will be soon, releasing it shortly to the ASF repo and then its just a maven-central sync cycle away. Note that if you tried the build with the patch I put in the vote email then you actually already have it installed in your local repo :) > TBH I've been avoiding touching this on trunk whilst it was in a state of > flux 'cause I didn't want to add to the carnage, so it'll be good to get > things onto trunk and hopefully have people playing to help make sure it's > nicely bedded down and stable again. > > Looking at the recent Jiras am I correct that you've made updates so that > it (the instance on the maven branch) is in sync with the recent Java > Broker changes? > > Yes, though as Rob said, compilation only. Ensuring it actualy works is still to come later, which doing this would help facilitate. > I doubt I'll have a chance to play with this until next weekend at the > earliest 'cause I've got family stuff planned pretty much all this weekend. > > Is there a chance of problems due to me having tried things out on the > branch previously - in other words is it sensible to blat my .mvn and start > from scratch or should all the versioning stuff in the various poms make it > all behave nicely? > > Cheers, > Frase > > I have updated the build itself so that the qmf bits depend on the current trunk/0.30-SNAPSHOT broker. You should see no issues from having tried things previously on the branch, as the qmf bits always had 0.30-SNAPSHOT version themselves on the branch but just depended on the then-current trunk 0.28-SNAPSHOT broker, which maven will now ignore and use the 0.30-SNAPSHOT artifacts, either grabbing from the snapshots repo or finding in your local repo from your doing 'mvn install' in the main java tree. (sidenote: you wouldnt need to nuke the whole ~/.m2 folder in any case if it was messed up, just removing the subtree org/apache/qpid containing our artifacts would be sufficient...but you should rarely ever need to do that) > > > On 02/05/14 11:03, Robbie Gemmell wrote: > >> Ok, I'd like to merge the maven build for the QMF bits to trunk now for a >> couple of reasons: >> - The existing Ant build (in addition to the code) has presumably been >> broken by recent build related changes made in the main java tree on >> trunk. >> - When I release the initial qpid-parent pom in the coming hours it should >> now all 'Just Work'* (using the remote snapshots unless you install a >> local >> version using the maven build in the main java tree). >> >> Robbie >> >> * Where for now 'Just Work' = compile, Rob etc will no doubt still need to >> further check/fix the impact on actual functionality of changes they >> have/are going to make. Doing this will make it easier for them to do >> that. >> >> On 6 April 2014 18:09, Fraser Adams >> wrote: >> >> Thanks again for all the responses Robbie. >>> >>> Go for it. >>> >>> If I get burned too badly I can always find out where you live >>> mwa mwa mwa mwa!! >>> >>> Frase >>> >>> >>> >>> On 06/04/14 15:50, Robbie Gemmell wrote: >>> >>> On 6 April 2014 10:29, Fraser Adams >>>> wrote: >>>> >>>> Hi Robbie, >>>> >>>>> Firstly thanks so much for taking the time to give such a comprehensive >>>>> response. I'll summarise my current thoughts below. >>>>> >>>>> 1. The directory structure in qpid/tools/src/java seems fine, I've got >>>>> no >>>>> strong aversion to it. There might be some tweakage, and possibly as >>>>> you've >>>>> suggested before there might be mileage in separating into console, >>>>> agent, >>>>> common+util as most applications using it would only tend to need the >>>>> console, common and JMS stuff, but it's probably not worth doing too >>>>> much >>>>> there at the moment 'cause I've no idea what carnage I'll cause when I >>>>> start to (eventually) get round to doing AMQP 1.0 Management - quite >>>>> how >>>>> I >>>>> handle any migration is very much TBD :-/ >>>>> >>>>> 2. I can see how Maven can (sort of) simplify build scripts, though >>>>> that >>>>> only seems to apply to large projects like the Java Broker, for the QMF >>>>> stuff, which is ultimately just a JMS client it seems to have gone from >>>>> one >>>>> fairly simple build.xml which was pretty straightforward to something >>>>> like >>>>> six pom.xml files - there's progress for you :-D >>>>> >>>>> >>>>> You only have to deal with the top level pom.xml generally when >>>>> building >>>>> >>>> stuff, sure there may be more files 'behind' that but you basically dont >>>> need to look at them unless you want to change them, at which point I >>>> think >>>> it actually becomes easier to find what you want to change because >>>> things >>>> are nicely split up into modules that deal with themselves, and a parent >>>> that everything inherits. >>>> >>>> We could have left the non-broker-plugin stuff as one big aggregate jar, >>>> but the reality is that the existing 'qmf.jar' contains a lot more than >>>> a >>>> single thing and so splitting it up to at least some extent seemed like >>>> the >>>> correct thing to do. The python/original QMF implementation and command >>>> line tools built around it are separated in a similar way, presumably >>>> for >>>> similar reasons. >>>> >>>> >>>> 3. In the qpid-qmf2-parent pom I don't really understand the >>>> >>>>> dependencies, >>>>> that pom seems to be responsible for the top-level building of the >>>>> modules, >>>>> but why does it have dependencies of its own (log4j, slf4j, jms spec)? >>>>> I >>>>> would get it if this had the common dependencies of all the >>>>> sub-modules, >>>>> but the qpid-qmf2 pom has its own dependencies on slf4j and jms spec - >>>>> exposing more holes in my lack of understanding of Maven I guess. >>>>> >>>>> First up, as with qpid-parent yesterday, qpid-qmf2-parent actually >>>>> has >>>>> >>>> no >>>> dependencies, but it does have a 'dependency management' section, which >>>> is >>>> used to ensure consistent alignment of dependency versions in the child >>>> modules and make their poms slightly simpler. See >>>> http://maven.apache.org/guides/introduction/introduction-to-dependency- >>>> mechanism.html#Dependency_Management >>>> >>>> Next up, inheritance. The parent pom uses 'pom' packaging and doesnt >>>> produce any output except itself, while the individual modules then use >>>> it >>>> as their parent and then inherit much of its contents - such as the >>>> dependencyManagement section that in turn ensures they all use the same >>>> versions for those things going forward. >>>> >>>> I could have left out the dependencyManagement section and just repeated >>>> the SLF4J and JMS spec dependency etries in their entirity in every >>>> module >>>> requiring them, but that becomes harder to maintain going forward and is >>>> more prone to becoming inconsistent. >>>> >>>> Some of this information is overriding bits from qpid-parent pom, which >>>> currently contains a much larger amount of dependency management, but we >>>> are considering moving some of that back to its immediate children as it >>>> doesnt all apply universally to them. I also now realise that a (highly >>>> unusual going forward) change I made yesterday to qpid-parent might not >>>> sit >>>> nice with the dependencyManagement section in qpid-qmf2-parent so I'll >>>> go >>>> fix that if required - which yes it was, update commmited :) >>>> >>>> >>>> 4. The main benefit that I can see of Maven, aside from (sort of) >>>> >>>>> simplifying build scripts (which only seems to apply for large builds) >>>>> is >>>>> that it does seem to provide useful artefacts from the perspective of >>>>> *users*. That is to say it's biggest (or only as far as I can see ;->) >>>>> benefit is from the perspective of release packaging - you end up with >>>>> some >>>>> nice tar.gz files that contain what's needed, so I sort of get why it's >>>>> nice from a user perspective - maybe that's worth the price of >>>>> admission >>>>> alone. >>>>> >>>>> We always had the ability to make nice tar.gz files, its just much >>>>> more >>>>> >>>> of >>>> a pain in the ass to do so in our Ant build, often requiring significant >>>> hoop jumping and typically coming out during the release process that >>>> they >>>> were wrong because we dont actually use them the rest of the time (due >>>> to >>>> the convenient but horrible global build dir that doesnt represent what >>>> users actually get). >>>> >>>> For me the main benefits are the consistency (this obviously doesnt >>>> apply >>>> to you yet, but the many people who know maven from use on other >>>> projects >>>> will now largely be able to pick up the Qpid build and use most of it >>>> without assistance, while the same wasn't really true of the Ant build >>>> because so much of it is bespoke) and extending from that the ease of >>>> integration with a lot of tooling that for the most part works simply >>>> with >>>> or is actually built into maven builds but usually requires pulling >>>> teeth >>>> (often mine) to get working with highly bespoke Ant builds such as ours. >>>> >>>> It will also mean we (i.e me) dont need to make a special effort to >>>> publish >>>> maven artifacts for components to maven central after our users request >>>> it >>>> (which they wont need to because we just will), as we (i.e me) always >>>> have >>>> in the past, or waste considerable amounts of [my] time verifying those >>>> generated artifacts actually contain what they should every time we >>>> release >>>> (which they often dont - I have yet to check them this time round) >>>> because >>>> we dont actually use them the rest of the time. >>>> >>>> >>>> 5. From a *dev* perspective though, at the moment the bit of the Maven >>>> >>>>> rework that a dislike the most is more or less exactly the same as >>>>> point >>>>> 4. >>>>> above. In order to actually use the Java Broker I have to do mvn clean >>>>> install, >>>>> >>>>> You just need to run mvn package currently. You only need to use >>>> install >>>> if >>>> you want to build something entirely separate using maven which declares >>>> it >>>> depends on the artifacts for the broker you just built. Strictly >>>> speaking >>>> you only need actually do that if you want to modify the broker first, >>>> as >>>> it will otherwise download the release or snapshot version (based on >>>> your >>>> specified dependency) from the remote repositories if you dont have it >>>> in >>>> your local repo already. >>>> >>>> >>>> I then need to take the tar.gz from /java/broke/target copy it >>>> >>>>> somewhere and unzip it - inconvenient! >>>>> >>>>> As mentioned yesterday, its simply not finished. Removing this need >>>> is a >>>> 'nice to have' thing in my mind, which has ranked it below some other >>>> 'needs to be done before this is correct / suitable for use at all' >>>> things. >>>> >>>> I have basically viewed this personally along the lines of....until we >>>> do >>>> put something into the poms to start or unpack the broker, we actually >>>> have >>>> to use/do something a user e.g. looking at the website would do? Oh the >>>> horror ;) >>>> >>>> >>>> But that's not the worse bit, when I need to figure out what's broken >>>> >>>>> next >>>>> time the plugin API changes there's a whole world of copying and >>>>> unzipping >>>>> to be had, which as well as being inconvenient is prone to finger >>>>> trouble. >>>>> >>>>> Build and install the broker artifacts (or dont and it will just use >>>> the >>>> latest snapshot). Build the QMF bits, which will fail if we broke >>>> something. Dont touch the tar file, or any jar files, or set the >>>> classpath. >>>> This stage is far easier than it was before with the Ant build for the >>>> QMF >>>> bits. >>>> >>>> When it comes to running it all up to make sure it works, yes you will >>>> need >>>> to copy some things *currently*. That could certainly be done via config >>>> in >>>> the poms if needed to 'automate' it. Its probably worth noting that >>>> having >>>> some/any real automated tests for these bits would mean that this step >>>> wouldnt even be necessary much of the time. >>>> >>>> A key thing I think is being overlooked is that we will simply try >>>> harder >>>> not to break it in future. Because it wont be a complete pain to build >>>> as >>>> it is now, we will far more easily be able to ensure it is kept up to >>>> date. >>>> We would also now be able to very easily stick it in CI to make sure >>>> that >>>> we know we broke it if we hadnt noticed in advance (which I expect we >>>> will >>>> more, because it wont be an ordeal to do so anymore). >>>> >>>> >>>> It's unfortunate that Maven install only "installs" things to the >>>> >>>>> repository and that there doesn't seem to be a way to install assembly >>>>> artefacts onto my system. I guess I could write my own private script >>>>> that >>>>> does all that - but it is as I say inconvenient from a dev perspective. >>>>> >>>>> Thats the primary purpose of mvn install. We can however bind >>>>> anything >>>>> >>>> we >>>> like to any of the build phases, perhaps adding a profile that unpacks >>>> to >>>> a >>>> specified directory as a 'system installation' type command etc etc. As >>>> I >>>> said yesterday, there are a multitude of options here, it just isnt >>>> something we have got to *yet* because it really is only a convenience >>>> whereas other things are not. >>>> >>>> >>>> 6. Hmmm so I say item 5 was what I dislike the most, actually one >>>> thing I >>>> >>>>> dislike just a bit more is what it does for WebApp development. What I >>>>> really like about programming in JavaScript is that I can just make a >>>>> change, hit refresh and bingo, none of this build/compile malarkey. Now >>>>> from a user perspective it's great that the QMF WebApp is neatly >>>>> packaged >>>>> in qpid-qmf2-tools.tar.gz but from a dev perspective it's a car crash >>>>> waiting to happen :-( In order to run QpidRestAPI I have to unpack the >>>>> tools, but that will then use the packaged WebApp by default, now there >>>>> is >>>>> a command line option so I can point webroot to >>>>> /tools/src/java/qpid-qmf2-tools/bin/qpid-web >>>>> on trunk, so it's not like I can't work around it, but how long do you >>>>> give >>>>> me before I start accidentally making changes in the "installed" >>>>> version >>>>> - >>>>> then accidentally lose them all next time I update the QpidRestAPI. I >>>>> might >>>>> be lucky, but I have a very simple brain :-) >>>>> >>>>> You could probably make a profile to it run the GUI straight out of >>>>> the >>>>> >>>> src >>>> tree without ever building the package. Thats basically how the 'test' >>>> classes work for example since it doesnt even need to build the jar >>>> files >>>> to run them. >>>> >>>> >>>> 7. I'm a bit worried about the Maven version stuff. I can see it >>>> being a >>>> >>>>> good thing wrt. being able to do reproducible builds, but does it >>>>> really >>>>> mean that every time a new Qpid release comes out there will be a need >>>>> to >>>>> try and remember to go through dozens of pom.xml files to update >>>>> various >>>>> artefact versions? Given that there are six poms for the QMF stuff >>>>> alone >>>>> that doesn't feel brilliant. >>>>> >>>>> There are plugins that can help with this. >>>>> >>>> >>>> 8. It feels like I'm pushed into Mavenising every other little Qpid >>>> test >>>> >>>>> app I've got on my system. Before I had things (albeit perhaps messily) >>>>> on >>>>> my CLASSPATH, but now not. >>>>> >>>>> As you say, this new way is cleaner, but you cancertainly do basically >>>> what >>>> you did before if you like, you will just have to specify more locations >>>> in >>>> your classpath. >>>> >>>> For production things that's fine and the Maven packaging does seem >>>> quite >>>> >>>> good, but for all the other hacky little things .... sigh, I really do >>>>> think Maven is a virus, once one thing catches it you have to "update" >>>>> everything else. >>>>> >>>>> You arent really forced in any way in my mind. Sure, it makes >>>>> something >>>>> >>>> you >>>> are doing slightly uglier, but you are entirely free to keep doing that >>>> hacky thing if you like. >>>> >>>> I feel a related consideration also needs to be applied in the opposite >>>> direction, i.e the folks (lots of them) that have long had maven builds >>>> and >>>> might come along and want to use the output of our Ant build. Poor sods. >>>> >>>> >>>> 9. All that said, there seems to be an inexorable trend towards >>>> Maven, so >>>> >>>>> from my perspective you might as well move forward on the great Qpid >>>>> Mavenisation. I'm not going to stand in the way even if I might moan a >>>>> bit >>>>> until I get used to it :-D. >>>>> >>>>> Feel free to moan away :) >>>> >>>> It'd be nice though if there were some solutions to the productivity >>>> issues >>>> >>>> >>>> I realise that this transition might involve a temporary productivity >>>> dip >>>> for some who arent as used to it already, but also consider the >>>> productivity increase others will get from it (I'll be honest: whilst I >>>> think that does apply generally, I do primarily mean me, who wont have >>>> to >>>> spend so much time rewriting chunks the Ant build to do really simple >>>> things, or waste as much time verifying the output we rarely generate is >>>> actually correct, or ..) and that a key driver >>>> is >>>> allowing us to progress towards making the project as a whole more >>>> component focused, which will itself improve productivity and wouldnt be >>>> possible without rewriting what we had anyway. We will get to the little >>>> things like starting the broker one its built. >>>> >>>> There is obviously a learning curve, but its not that big, once you are >>>> over it I actually think you would actually come to see increased >>>> productivity from using it generally instead of writing Ant scripts (or >>>> dealing directly with javac) and tinkering with the CLASSPATH for >>>> example. >>>> >>>> >>>> - I already struggle with productivity given limited coding time >>>> between >>>> >>>>> work and family stuff, some weekends I feel like I've wasted it all >>>>> chasing >>>>> my tail rather than doing anything useful. >>>>> >>>>> Regards, >>>>> Frase >>>>> >>>>> >>>>> >>>>> >>>>> On 05/04/14 18:57, Robbie Gemmell wrote: >>>>> >>>>> On 4 April 2014 14:41, Fraser Adams >>>>> >>>>>> wrote: >>>>>> >>>>>> On 04/04/14 13:30, Rob Godfrey wrote: >>>>>> >>>>>> No... "install in the local repo" is what the mvn clean install >>>>>>> will >>>>>>> >>>>>>> do... So, you literally just have to follow Robbie's instructions: >>>>>>>> check >>>>>>>> out the qpid-parent-pom/trunk directory, run mvn clean install from >>>>>>>> wherever you checked it out to... then go back to the broker and the >>>>>>>> maven >>>>>>>> build should complete ok. Hope this helps, Rob >>>>>>>> >>>>>>>> Cheers Rob, >>>>>>>> >>>>>>>> I ended up figuring out that myself after staring at the >>>>>>> instructions >>>>>>> for >>>>>>> a while and deciding that I was reading way too much into things :-) >>>>>>> >>>>>>> I've now got the Java Broker etc. built using Maven but before I play >>>>>>> with >>>>>>> the QMF things I wouldn't mind answers to a few Maven questions - I'm >>>>>>> still >>>>>>> very much at the "Burn the Witch" stage wrt. my trust of Maven ;-) >>>>>>> >>>>>>> I've now ended up with a directory >>>>>>> /home/fadams/.m2/repository >>>>>>> >>>>>>> Filled with stuff. >>>>>>> >>>>>>> I'm not clear by what witchcraft Maven decides what to shove there - >>>>>>> >>>>>>> Maven has the concept of a local repo and a remote repo, analogous >>>>>>> to >>>>>>> >>>>>> e.g. >>>>>> your installed packages in a linux distro, and the repository the >>>>>> package >>>>>> manager got them from. The default local repo localtion is >>>>>> ~/.m2/repository, and the default remote repo is maven central. >>>>>> >>>>>> Any dependencies it needs to download will go in the local repo, so it >>>>>> doenst necessarily need to download them again next time they are used >>>>>> (unles e.g they are snapshots and new versions came out). >>>>>> Additionally, >>>>>> anything you run "mvn install" on will go in the local repo configured >>>>>> at >>>>>> the time (can be overriden per-command for example) to allow it to be >>>>>> used >>>>>> by other things you build later. >>>>>> >>>>>> 'Any dependencies' above includes the dependencies of a particular >>>>>> artifact >>>>>> you are using, the [plugin] dependencies of anything needed by your >>>>>> build >>>>>> process to do what you are asking, and any [plugin] dependencies >>>>>> needed >>>>>> by >>>>>> maven itself. >>>>>> >>>>>> (Additional reading, there are different dependency scopes that can >>>>>> influence if/when they are needed/used etc: >>>>>> http://maven.apache.org/guides/introduction/ >>>>>> introduction-to-dependency- >>>>>> mechanism.html#Dependency_Scope >>>>>> ) >>>>>> >>>>>> >>>>>> for example I eyeballed the parent pom.xml and saw the >>>>>> dependencies, >>>>>> The parent pom actually has no real dependencies (except on its >>>>>> parent, >>>>>> the >>>>>> apache pom). >>>>>> >>>>>> What you likely saw was the pluginManagement and dependencyManagement >>>>>> sections, which help fix down and/or consistently specify the versions >>>>>> of >>>>>> particular dependencies and plugins that might be used by the child >>>>>> modules >>>>>> and their build steps. These dont actually mean things have a >>>>>> dependency >>>>>> on >>>>>> those artifacts, just which version etc they would get by default if >>>>>> they >>>>>> should happen to specify such a dependency or plugin to use. This >>>>>> helps >>>>>> keep things using consistent versions of artifacts, and in the case of >>>>>> plugins helps ensure reproducable builds by locking down the versions >>>>>> in >>>>>> use. >>>>>> >>>>>> >>>>>> but when I looked at the repository after running mvn clean >>>>>> install on >>>>>> >>>>>> that I was kind of expecting the directories in the repository to >>>>>>> pretty >>>>>>> much match what was in the dependencies, but it definitely didn't >>>>>>> after >>>>>>> running mvn on the parent pom.xml, though to be fair after building >>>>>>> the >>>>>>> main things it does all appear to match (though with a ton of other >>>>>>> stuff >>>>>>> too). >>>>>>> >>>>>>> What you would have got after doing 'mvn install' on the >>>>>>> qpid-parent >>>>>>> pom >>>>>>> >>>>>>> would be mostly only the plugins needed by maven itself to carry out >>>>>> those >>>>>> actions. What they ship in the core maven release is augmented by >>>>>> downloading/installing things to the local repo as they are used both >>>>>> by >>>>>> itself and by people declaring they want to use a specific plugin in >>>>>> their >>>>>> build. >>>>>> >>>>>> >>>>>> When I build the Java Broker etc. it seemed to take an age, when I >>>>>> was >>>>>> >>>>>> using ant it took under a minute on my system, but with Maven it >>>>>>> reports >>>>>>> a >>>>>>> total time of 5:18 min and it looks like it was downloading half the >>>>>>> Internet :-D I'm *guessing* that this is a one-off cost as it fills >>>>>>> up >>>>>>> my >>>>>>> local repo with stuff? >>>>>>> >>>>>>> >>>>>>> Yes, the first time you would have incurred the cost of grabbing >>>>>>> any >>>>>>> >>>>>>> plugins the build needed that you didnt already have and also the >>>>>> dependencies of the things being built (the later of which happens on >>>>>> your >>>>>> first Ant build too - go look in ~/.ivy2/cache, then try deleting it >>>>>> and >>>>>> see what happens to the Ant build time). Run it again and it will be a >>>>>> lot >>>>>> faster going forward. >>>>>> >>>>>> On my machine using the head of trunk (I just committed a bunch of >>>>>> tweaks >>>>>> earlier), 'ant clean build' vs 'mvn clean package -DskipTests=true' >>>>>> has >>>>>> the >>>>>> maven build being 1 or 2sec slower (32 vs 33/34), which is actually >>>>>> surprisingly close since using those commands we currenty have the >>>>>> maven >>>>>> build doing a bunch of extra packaging and enforcement that the ant >>>>>> build >>>>>> doesnt. It may also be quicker if told to build 'offline' to ensure it >>>>>> didnt check any repos for updates. >>>>>> >>>>>> (It takes a further 2mins 21sec on top of that for me to use the Ant >>>>>> build >>>>>> to generate the maven artifacts we currently publish to central...a >>>>>> generation step we obviously wont need to do for the maven build as >>>>>> they >>>>>> are already there inherantly.) >>>>>> >>>>>> >>>>>> Where do you set CLASSPATH when you build using Maven? With the ant >>>>>> build >>>>>> >>>>>> I used to have: >>>>>>> qpid/java/build/lib/qpid-all.jar >>>>>>> >>>>>>> >>>>>>> You dont set CLASSPATH, basically. Supplying dependency >>>>>>> information >>>>>>> is >>>>>>> >>>>>>> all >>>>>> handled through the poms, which maven uses to build the classpath for >>>>>> each >>>>>> module as it builds. If you want to build something else that uses e.g >>>>>> the >>>>>> qpid client, the thing your are building should have a dependency for >>>>>> qpid-client in its pom. You can make that dependency be either a >>>>>> published >>>>>> release [or snapshot] version available in central [or the apache >>>>>> snapshots >>>>>> repo], or you can 'mvn install' your own modified copy if you want to >>>>>> use >>>>>> an unpublished client release. If you dont want to use maven / want to >>>>>> use >>>>>> basic javac commands for the other thing you are compiling against the >>>>>> client, then you would need to make the client jars available >>>>>> somewhere >>>>>> and >>>>>> add the individual jars to your classpath currently. >>>>>> >>>>>> We aren't recreating qpid-all at this time because while it may be >>>>>> convenient in cases, it is actually a horrible horrible thing, which >>>>>> I'll >>>>>> talk a bit more to later when covering the build directory :) >>>>>> >>>>>> We have a qpid-all file in every release artifact made by the ant >>>>>> build, >>>>>> each having different content, and all different from the one that >>>>>> gets >>>>>> created in the build dir. Even if we wanted to recreate it in the >>>>>> maven >>>>>> build, we would need to give it a unique name for every component to >>>>>> ever >>>>>> publish it anyway. >>>>>> >>>>>> We could instead do something like add the classpath manifest entries >>>>>> previously contained in qpid-all.jar into the main qpid-client, >>>>>> qpid-broker >>>>>> etc jars so you could do 'java -jar qpid-foo' etc on them instead. >>>>>> >>>>>> >>>>>> Which was nice and convenient, if possibly a bit sloppy, from what >>>>>> I >>>>>> can >>>>>> >>>>>> see each qpid/java subdirectory seems to have a target directory e.g. >>>>>>> qpid/java/client/target though in there there is >>>>>>> qpid-client-0.28-SNAPSHOT.jar which seems less convenient that >>>>>>> qpid-client.jar if I want to set up CLASSPATH in my .bashrc. Am I >>>>>>> missing >>>>>>> something? >>>>>>> >>>>>>> Another thing that I'm not clear on is that in the Maven repository >>>>>>> there >>>>>>> seems to be a bunch of jars installed - as an example >>>>>>> org/eclipse/jetty/jetty-websocket/8.1.14.v20131031/ >>>>>>> jetty-websocket-8.1.14.v20131031.jar. >>>>>>> In the "olden days" Qpid pulled in jetty and the Jar was available in >>>>>>> qpid/java/build/lib so when I was messing around with Jetty on >>>>>>> another >>>>>>> project I had it handily on my CLASSPATH - I can't seem to see jetty >>>>>>> anywhere now except in the Maven repository? So how does the Broker >>>>>>> see >>>>>>> it? >>>>>>> >>>>>>> The broker sees it because it says it has a runtime dependency on >>>>>>> the >>>>>>> >>>>>>> management-http module in its pom, and the management-http module >>>>>> says >>>>>> it >>>>>> has a compile time dependency on Jetty in its pom, and so maven then >>>>>> knows >>>>>> the broker transitively needs jetty. The actual jars never leave the >>>>>> local >>>>>> repo unless we instruct the build to do something that requires that >>>>>> (e.g. >>>>>> package a binary assembly containing it, or copying it somewhere) in >>>>>> the >>>>>> pom or you execute a command manually that does (e.g. a quick Google >>>>>> says >>>>>> 'mvn dependency:copy-dependencies' requests the dependency plugin to >>>>>> run >>>>>> its copy-dependencies goal, which create a dir in target/dependency >>>>>> with >>>>>> everything in it). >>>>>> >>>>>> The only reason the Ant build still puts things in the lib dir (by >>>>>> using >>>>>> Ivy to grab thigns from the maven central repo, i.e roughly the same >>>>>> thing >>>>>> Maven is doing to download things) is so that I didnt need to rewrite >>>>>> even >>>>>> more of the Ant build than I already was when I modified it to allow >>>>>> removing the jars from the lib dir, where they were actually committed >>>>>> into >>>>>> the svn repo historically (I refer you to my previous mails comment >>>>>> about >>>>>> the Ant build being a pain in the ass and me being the one best placed >>>>>> to >>>>>> speak best to this point :P). >>>>>> >>>>>> >>>>>> >>>>>> In the olden days when I just did "ant" at the end of that I ended >>>>>> up >>>>>> >>>>>> with >>>>>>> something that would run 'cause I had >>>>>>> qpid/java/build/bin >>>>>>> >>>>>>> on my path and could simply do "qpid-server" but now there's no nice >>>>>>> convenient build directory. >>>>>>> >>>>>>> The existing build directory is horrible and should burn :P >>>>>>> >>>>>>> I'll admit, it can be convenient, but its also a complete jumble >>>>>> with a >>>>>> little bit of extra mess on top, mixing up all the different >>>>>> components >>>>>> together in lib so you cant tell which bits need what (qpid-all.jar in >>>>>> the >>>>>> build directory being worst of all, as you have everything on the >>>>>> classpath >>>>>> manifest in there) and doing ugly copies of stuff into subdirs of lib >>>>>> to >>>>>> seperate them for later use precisely because you cant tell which bits >>>>>> need >>>>>> what. I would have tried to kill it a long time ago if it didnt >>>>>> involve >>>>>> yet >>>>>> more 'rewriting the ant build' fun, which I just couldnt bring myself >>>>>> to >>>>>> subject me to :) >>>>>> >>>>>> Maven deals with modules, which each have their own target dir for >>>>>> their >>>>>> intermediate work and final output, and if you want something that >>>>>> aggregates multiple modules then there are varous ways to do that, >>>>>> avoiding >>>>>> the global build dir mess while still letting you compose a larger >>>>>> unit >>>>>> from the individual bits. It is certainly different than what our Ant >>>>>> build >>>>>> does, but in this case I think its a good thing. We can always do >>>>>> things >>>>>> to >>>>>> make life easier, more below. >>>>>> >>>>>> >>>>>> I noticed in >>>>>> >>>>>> qpid/java/broker/target/qpid-broker-0.28-SNAPSHOT-bin.tar.gz that >>>>>>> archive >>>>>>> seems to contain qpid-server, do I *really* have to now have to faff >>>>>>> around >>>>>>> copying and unpacking archives everytime I want to update a build? >>>>>>> That >>>>>>> seems an awful lot less convenient than simply doing "ant clean" then >>>>>>> "ant". >>>>>>> >>>>>>> Have I missed something? I hope so 'cause if not it seems a step >>>>>>> backwards >>>>>>> from a "just works" POV >>>>>>> >>>>>>> What you say is true *currently*, to run the broker built by the >>>>>>> maven >>>>>>> >>>>>>> build you would need to unpack that tar file. We can certainly add >>>>>> functionality to remove that need, we just havent got to those final >>>>>> few >>>>>> 'nice to have' things yet. There are a multitude of ways we could do >>>>>> it, >>>>>> for example we could use a plugin and have the maven build itself >>>>>> start >>>>>> the >>>>>> broker (e.g see what i did with the 'tests' in the QMF tree: those >>>>>> actually >>>>>> run without even creating a jar file), or add some build config to >>>>>> allow >>>>>> output of an unpackaged version of it during the main build, >>>>>> or..>>>>> idea>. >>>>>> >>>>>> >>>>>> This might all be second nature to folks familiar with Maven, but >>>>>> I'm >>>>>> a >>>>>> >>>>>> bit "old skool" and I quite like knowing what's installed on my >>>>>>> system, >>>>>>> where it is installed, and why it's there so I'd quite like a bit of >>>>>>> reassurance :-) >>>>>>> >>>>>>> Frase >>>>>>> >>>>>>> >>>>>>> >>>>>>> ------------------------------------------------------------ >>>>>>> --------- >>>>>>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org >>>>>>> For additional commands, e-mail: users-help@qpid.apache.org >>>>>>> >>>>>>> >>>>>>> >>>>>>> ------------------------------------------------------------ >>>>>>> --------- >>>>>>> >>>>>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org >>>>> For additional commands, e-mail: users-help@qpid.apache.org >>>>> >>>>> >>>>> >>>>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org >>> For additional commands, e-mail: users-help@qpid.apache.org >>> >>> >>> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org > For additional commands, e-mail: users-help@qpid.apache.org > > --047d7b2e7386bcfe6d04f8691378--