Return-Path: X-Original-To: apmail-aries-dev-archive@www.apache.org Delivered-To: apmail-aries-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 43395D5BD for ; Fri, 29 Jun 2012 09:23:46 +0000 (UTC) Received: (qmail 83082 invoked by uid 500); 29 Jun 2012 09:23:46 -0000 Delivered-To: apmail-aries-dev-archive@aries.apache.org Received: (qmail 83038 invoked by uid 500); 29 Jun 2012 09:23:45 -0000 Mailing-List: contact dev-help@aries.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@aries.apache.org Delivered-To: mailing list dev@aries.apache.org Received: (qmail 83024 invoked by uid 99); 29 Jun 2012 09:23:45 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 29 Jun 2012 09:23:45 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of holly.k.cummins@googlemail.com designates 74.125.82.48 as permitted sender) Received: from [74.125.82.48] (HELO mail-wg0-f48.google.com) (74.125.82.48) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 29 Jun 2012 09:23:38 +0000 Received: by wgbdq11 with SMTP id dq11so2206563wgb.17 for ; Fri, 29 Jun 2012 02:23:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20120113; h=message-id:from:to:in-reply-to:content-type :content-transfer-encoding:x-mailer:mime-version:subject:date :references; bh=KlXeEduL4a6x7ORAW0E1cYQ8h72GaCZBtgaqa0FGXpI=; b=fk3cFsmsN9p/3qgeQT90GrHbNKfx0dd08wP+/QAUd3u+XPyYd3Y1Krx+2GgDo4h7jk umBaMJq8RzXovoBUoi5eFL+sxaBoPiRHeC+KIMU7sMyo6SalD0tsEJtNYeqnHHFNL+YT UR3ouzp9lRlVYHfEs1uMK2mbhEAptfsL4UsDF+R4IUDjpUd0/4ET+AkEjLir/ONTqlaQ AqVaxKHUqOBS14wBUYW2DGUVv9ZF8OvNj9AOmF9WHZ+Rll5q1JoSjLeR1Uw+l4kdu7nv 7PZML8eDj4vVoiX0ZuOZpradXPHeouk8WTu06hCHjKqFK41M0RdB9JznbDekHKeaR/vs vjiQ== Received: by 10.180.79.166 with SMTP id k6mr2927774wix.8.1340961796741; Fri, 29 Jun 2012 02:23:16 -0700 (PDT) Received: from [192.168.1.111] (188-222-130-142.zone13.bethere.co.uk. [188.222.130.142]) by mx.google.com with ESMTPS id eu4sm4703345wib.2.2012.06.29.02.23.14 (version=TLSv1/SSLv3 cipher=OTHER); Fri, 29 Jun 2012 02:23:16 -0700 (PDT) Message-Id: <10140881-8938-4E6F-936C-DEDD370B0D86@googlemail.com> From: Holly Cummins To: "dev@aries.apache.org" In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit X-Mailer: iPhone Mail (7E18) Mime-Version: 1.0 (iPhone Mail 7E18) Subject: Re: [DISCUSSION] Modifications to Aries release process Date: Fri, 29 Jun 2012 10:23:02 +0100 References: <2956591.HBzuJNjzLo@dilbert.dankulp.com> X-Virus-Checked: Checked by ClamAV on apache.org Right, it's that compile-order dependency which I'm hoping to avoid, since with 118 bundles, it's a lot of things to get in the right order - my hope was that it would be easier for PMC members to validate releases if each one was trivial to compile. I'll push on with the release of the util bundle, since that's the one *everything* depends on, and hopefully after that I'll be dealing with mire decoupled bundles, which means I'll be more able to release more meaningful chunks of function. If people would prefer to vote on fewer bigger releases at the cost of internal compile-order dependencies, I can certainly do it that way, although I do think all 118 bundles in one go would still be too much for the PMC to digest. :) On 29 Jun 2012, at 08:28, Guillaume Nodet wrote: > They would be compilable, you just have to compile them in the right > order > from the tags / source distributions. > If you (the release manager) have been able to compile them, anyone > who > uses the same tags will be able to do so too. > > On Thu, Jun 28, 2012 at 11:18 PM, Holly Cummins < > holly.k.cummins@googlemail.com> wrote: > >> Wouldn't the bundles be testable, but not compile-able from source, >> since the 1.0.0 dependencies would only exist in the staging >> repository? Certainly if we could do everything in one shot it would >> make the process of getting everything out the door much faster ... >> >> On Thu, Jun 28, 2012 at 6:12 PM, Guillaume Nodet >> wrote: >>> I think you made a wrong assumption which is that the staging repo >>> has >>> to be backed by a single svn revision (which is btw not really the >>> case because it comes from several tags anyway). >>> What I'd suggest is doing the api bundles release and have them >>> uploaded to nexus, then upgrade the implementations and all other >>> components to use those releases, commit, release those other >>> projects, THEN, close the staging repo and call for a vote. >>> So we'd have a single staging repo / vote with a set of testable >>> bundles >> in it. >>> >>> On Thu, Jun 28, 2012 at 6:45 PM, Holly Cummins >>> wrote: >>>> Hi Guillaume, >>>> >>>> Thanks for your comments. Here are some of my thoughts ... >>>> >>>> On Thu, Jun 28, 2012 at 3:19 PM, Guillaume Nodet >> wrote: >>>>> I think one problem, considering the current votes, is that it's >>>>> really difficult to test anything. Releasing api bundles with no >>>>> implementation to test is definitely not helping imo. >>>> >>>> I know what you mean about the testing, and I'm not totally sure >>>> what >>>> the best answer is. I know what I'm releasing comes from trunk, >>>> and is >>>> being tested by the Jenkins builds, so I'm pretty confident it >>>> works >>>> in a real system. However being tested in more environments and by >>>> more systems is obviously a Good Thing. I think the best way to >>>> test >>>> the API bundles is with the current -SNAPSHOT bundles of the >>>> implementation, either in something like the blog sample or some >>>> other >>>> working system. If we weren't moving from 0.x to 1.0 you could also >>>> test micro releases alongside existing impl bundles to ensure >>>> everything resolves and works as claimed. >>>> >>>>> Holly, just a question: is there a specific reason why are you >>>>> doing >>>>> the release in multiple votes ? It would be simpler to just >>>>> release >>>>> everything in one go and wait for a longer time because there >>>>> are more >>>>> things to check, or at least, release the api + implementation >>>>> so that >>>>> we can actually try something. Just my 2 cents. >>>> >>>> I agree that this sort of 'extended incremental' release is a bit >>>> awkward, and I was wondering when someone would ask what on earth I >>>> was doing :). IMO it's the cleanest way to work with with >>>> release-by-bundle (which I know you disagree with). If I release >>>> everything in one go, there's a problem with the dependencies >>>> between >>>> bundles. At the moment in trunk, almost every dependency is a >>>> SNAPSHOT >>>> dependency. In the past we've updated all bundles to use non- >>>> SNAPSHOT >>>> (but not yet released) versions in a branch, and I could even do >>>> something similar without using a branch by briefly having the >>>> trunk >>>> builds produce 1.0.0 artefacts. However, I think this creates a >>>> greater burden for testers. If there are compilation-order >>>> dependencies between parts of a release which don't share a top- >>>> level >>>> pom, everyone verifying a script has to compile them in the right >>>> order. I count 118 bundles to release, so that's a lot of bundles >>>> to >>>> get in the right order, and I didn't think any PMC member would >>>> want >>>> to try. :) I guess this could be automated with a verification >>>> script >>>> which either hardcodes or calculates the dependency graph, but it >>>> seemed to me like more work for everyone and more risk for the >>>> release. My hope was that if verifying individual mini-releases was >>>> easy enough, doing multiple ones wouldn't be a problem (and in fact >>>> would nicely distribute any effort, making it easier to vote). >>>> >>>> I know at this stage some of you are thinking "and *this* is why >>>> release by bundle is a bad idea!", and that's not really a debate I >>>> want to re-open. Among other things, I think any re-engineering >>>> of our >>>> poms at this stage will further delay the release. >>>> >>>> The good news is I believe this problem will almost entirely go >>>> away >>>> for 1.0.x and 1.x releases, because the impl bundle will, in most >>>> cases, depend on an *already* released version of its API bundle or >>>> another Aries component. This means a bunch of related bundles >>>> could >>>> be released at the same time, without compile issues, or a >>>> meaningful >>>> release really could consist of just a single bundle. That's true >>>> modularity and it should give both us and our users big benefits. >>>> >>>> >>>>> >>>>> On Mon, Jun 25, 2012 at 8:07 PM, Daniel Kulp >>>>> wrote: >>>>>> >>>>>> Honestly, with the change to using Nexus, the SHA1 and MD5 >>>>>> checks are >>>>>> completely pointless. Nexus generates them itself based on >>>>>> what's >>>>>> uploaded. The "is it a valid signature" part of the GPG >>>>>> testing is >> also >>>>>> pointless as Nexus won't let you close the repo unless the >>>>>> signatures >> are >>>>>> valid. The only check you really need to do is to make sure >>>>>> the key >> that >>>>>> was used is "trusted" by you. (aka: was it really Holly who >> deployed those >>>>>> artifacts) So the monontonous parts of checking that stuff is >> really >>>>>> irrelevant at this point. (providing we trust that infra has >>>>>> Nexus >>>>>> sufficiently locked down and secure) >>>>>> >>>>>> >>>>>> I actually don't have a big issue with the difficulting in >>>>>> getting >> votes. >>>>>> I'm involved in another community that has a PMC that is easily 4 >> times the >>>>>> size of this one, yet we still have difficulting getting votes >>>>>> there. >>>>>> While not ideal, life events can cause priority shifts and such >>>>>> so >> people >>>>>> may not be able to be as responsive. >>>>>> >>>>>> My bigger problem is that the entire per bundle release process >>>>>> and >> symantic >>>>>> versioning crap has put a HUGE burden on the release manager. >>>>>> That >> makes >>>>>> it much harder to get quality releases out and makes it less >>>>>> likely >> that >>>>>> anyone will step up to get "minor fixes" released. The only >>>>>> reason I >>>>>> stepped up with the 0.3.1 bp stuff is that *MY* customers are >>>>>> being >>>>>> affected by it. Like wise for the proxy stuff. If *my* >>>>>> customers >> were >>>>>> not affected, I don't think I would have spent the time and >>>>>> effort. >> If >>>>>> the process for getting fixes and releases out to users was >>>>>> smaller >> and >>>>>> easier, I have no problem doing them. For CXF, we do full >>>>>> releases >> on 3 >>>>>> branches every other month or so. But that's because it's >>>>>> EASY to >> do. >>>>>> >>>>>> If it was up to me, I'd toss out the entire versioning thing >>>>>> with 1.0 >> and go >>>>>> back to per module versioning thing. So my fix to proxy >>>>>> would have >>>>>> involved checking out all of "proxy", fixing it, and releasing >>>>>> all of >> proxy >>>>>> as a proxy "0.3.1", even the modules that haven't changed. It's >> just a >>>>>> huge hassle to track down which bundles have changed, which >>>>>> haven't >> which >>>>>> version numbers need to be updated, etc.... If it's not quick >>>>>> and >> easy to >>>>>> do releases as a release manager, very few people are going to >>>>>> step >> up to do >>>>>> it. It may not be 100% "proper OSGi", but IMO, getting >>>>>> fixes and >> such to >>>>>> the users is more important than that. But that's my opinion. >>>>>> >>>>>> >>>>>> Dan >>>>>> >>>>>> >>>>>> >>>>>> On Saturday, June 23, 2012 03:27:07 PM Holly Cummins wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> Now that Jeremy's taken the time to write up our release >>>>>>> verification >>>>>>> process, I'd like to propose we change it. :) I think it's too >> onerous >>>>>>> on the pmc, which therefore also inhibits our ability to be >> responsive >>>>>>> to our users. >>>>>>> >>>>>>> >>>>>>> ------------------------------- Why what we have isn't working >>>>>>> for >> the >>>>>>> community ------------------------------- >>>>>>> >>>>>>> I believe our users would like more frequent releases. We've had >>>>>>> several keen requests and tweets and comments on the aries-user >>>>>>> mailing list wishing we'd release more often. For example: >>>>>>> >>>>>>> * "Desperately waiting for an Aries release after loooong >>>>>>> time.." >>>>>>> * "The problem with Aries is they seem to be too busy coding to >>>>>>> release anything." >>>>>>> * "Compared to other projects (like Karaf and Camel) Aries >>>>>>> releases >>>>>>> tend to take quite some time." >>>>>>> * "It's 2012 now and Aries 0.3 is almost a year old. Is there >>>>>>> any >>>>>>> chance of a new Aries JPA release any time soon? " >>>>>>> * "Looks like Apache Aries has made no visible progress since >>>>>>> Jan >>>>>>> 2011, if the time stamps on the maven central artefacts are to >>>>>>> be >>>>>>> believed." >>>>>>> >>>>>>> ------------------------------- Why what we have isn't working >>>>>>> for us >>>>>>> ------------------------------- >>>>>>> >>>>>>> Both Dan and I are trying to do releases at the moment, and >> struggling >>>>>>> to get enough PMC votes. Dan's release is to back port a show- >>>>>>> stopper >>>>>>> proxy fix, so a release there is particularly pressing - he's >>>>>>> got a >>>>>>> non-binding +infinity vote, but that's all. My test support >>>>>>> release >>>>>>> vote has been open for about 64 hours, and only got one vote >>>>>>> so far >>>>>>> (thanks David B!). Obviously testsupport is less exciting than >>>>>>> proxy, >>>>>>> but that bundle does block more interesting releases. >>>>>>> >>>>>>> Why aren't people voting? My guess is that it's too much work >>>>>>> to do >>>>>>> the full set of verifications described at >>>>>>> http://aries.apache.org/development/verifyingrelease.html. >>>>>>> There are >>>>>>> seven steps, and while they don't actually take that long to >> complete, >>>>>>> it's enough of a burden that we tend to leave the voting to >>>>>>> someone >>>>>>> else unless we really care about a release. I'm as guilty of >>>>>>> this as >>>>>>> anyone - I think a release is a good idea, but I'm spending >>>>>>> enough >>>>>>> time working on the 1.0.0 release that I don't want to take >>>>>>> time out >>>>>>> to vote on another release. I suspect Dan might feel exactly >>>>>>> the same >>>>>>> about my 1.0.0 bundles. :) >>>>>>> >>>>>>> With release-by-bundle, there's a lot of verifications. >>>>>>> Excluding the >>>>>>> sandbox code, we have 123 bundles to release in 1.0.0. At >>>>>>> three votes >>>>>>> per bundle, that means the PMC need to do 369 MD5 checks, 369 >>>>>>> PGP >>>>>>> checks, 369 RAT checks, and so on, just to get 1.0.0 out the >>>>>>> door. >>>>>>> This just doesn't seem like it scales. Batching the bundle >>>>>>> releases >>>>>>> together eases some of this burden, but not all. >>>>>>> >>>>>>> ------------------------------- What I propose >>>>>>> ------------------------------- >>>>>>> >>>>>>> I suggest we move to a more trust-based system, where PMC >>>>>>> members >>>>>>> carefully check releases if they want, but where in general >>>>>>> they're >>>>>>> voting on the principle of the release, rather than the >>>>>>> mechanics of >>>>>>> the archives. In particular, they don't feel compelled to do >>>>>>> checks >>>>>>> before voting. If PMC members could say "Our users need this >> function, >>>>>>> so +1", or "I know Holly has done sensible things in the past, >>>>>>> so +1" >>>>>>> or even "Do I want to check the SHAs on a test support bundle? >> Really? >>>>>>> +1" it would get our releases moving better, and also save >>>>>>> work for >>>>>>> all of us. >>>>>>> >>>>>>> (At the moment I think what's happening is people are thinking >>>>>>> "Do I >>>>>>> want to check the SHAs on a test support bundle? Really?" and >>>>>>> then >>>>>>> skipping the +1 bit. :) ) >>>>>>> >>>>>>> To ensure that at least *someone* has run the checks, the >>>>>>> release >>>>>>> manager could include the output of the seven checks in an >>>>>>> email to >>>>>>> the list. I think this level of checking is perfectly >>>>>>> compatible with >>>>>>> the minimum Apache process, which is that the release manager >>>>>>> signs >>>>>>> the artefacts and three PMC members vote +1 >>>>>>> (http://www.apache.org/dev/release-publishing.html#voted). >>>>>>> >>>>>>> What do people think? >>>>>>> >>>>>>> Holly >>>>>> -- >>>>>> Daniel Kulp >>>>>> dkulp@apache.org - http://dankulp.com/blog >>>>>> Talend Community Coder - http://coders.talend.com >>>>> >>>>> >>>>> >>>>> -- >>>>> ------------------------ >>>>> Guillaume Nodet >>>>> ------------------------ >>>>> Blog: http://gnodet.blogspot.com/ >>>>> ------------------------ >>>>> FuseSource, Integration everywhere >>>>> http://fusesource.com >>> >>> >>> >>> -- >>> ------------------------ >>> Guillaume Nodet >>> ------------------------ >>> Blog: http://gnodet.blogspot.com/ >>> ------------------------ >>> FuseSource, Integration everywhere >>> http://fusesource.com >> > > > > -- > ------------------------ > Guillaume Nodet > ------------------------ > Blog: http://gnodet.blogspot.com/ > ------------------------ > FuseSource, Integration everywhere > http://fusesource.com