geronimo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason van Zyl <>
Subject Re: no more modules for specs...
Date Sat, 16 Dec 2006 03:51:49 GMT

On 15 Dec 06, at 6:41 PM 15 Dec 06, Jason Dillon wrote:

> this change really killed me for all of the build automation I have  
> been working on... and was one of the reasons why I had warned  
> against using this style of versioning.
> basically since this was removed, checking out the specs project  
> and building it produces nothing, except for this:
>     org/apache/geronimo/specs/specs/maven-metadata-local.xml
>     org/apache/geronimo/specs/specs/1.2/specs-1.2.pom
>     org/apache/geronimo/specs/specs/1.2/specs-1.2-site.xml
> since my automation setup actually builds projects, then holds on  
> to the *exact* build output for use by a project which depends on  
> them, anything that depends on a spec which is not already released  
> and propagated to central and its mirrors will fail.  This means  
> that anything that needs a snapshot will always fail.
>  * * *
> one critical error I believe that everyone is making is thinking  
> that deployed artifacts are permanent...

Deployed releases that we place in the repository are most certainly  
permanent. Snapshots can be transient depending on the policy of the  
repository. What I've been told is that infrastructure is telling you  
to remove old versions from the repository at Apache which can have  
disasterous effects. Your artifacts deployed to central should not  
disappear so I'll go make sure that we're not deleting files that  
have been removed from the source. But we generally assume when we  
are syncing from a source that the source knows what it's doing. If  
you delete stuff on this end then we expect you to understand what  
effect that will have on your clients.

> which they are most certainly not, so we should generally always  
> build from source to ensure that everything is being included at  
> the right version.

No you shouldn't. That defeats the entire purpose of Maven. You have  
unmaintainable and non-reproducible builds because you have such a  
massive number of SNAPSHOTs and you never make any incremental  
releases. What do you expect when you're building upon sand? Get the  
producers of your artifacts to release things more frequently, this  
ensures that SNAPSHOTs are removed from the chain. I look in your  
server build and you've got SNAPSHOTS **all** over the place.  
OpenEJB, ActiveMQ, Tranql, your spec JARs. You control, or have  
relationships with most of these projects. You don't wait until it's  
time to release to get the producers of your dependencies to crap out  
some junk you can use. Get them release, lock your stuff down to it,  
move on. Repeat. You should have no SNAPSHOTs other then ones for  
your own projects in your POMs. If everything is changing everywhere  
it's total insanity. Build everything from source is not something  
you should be doing because you should not have to. It is one of the  
most obvious signifiers that you are doing something very wrong. You  
have way too many points of variation and all your projects seem to  
be in constant swing with one another. If that's the case it begs the  
question whether they really separate and why not just lump them all  
together if they are so coupled together. Which generally the case  
when you cannot eliminate SNAPSHOTs from your build. When you see  
SNAPSHOT not from your project in our POM it is a warning sign.

> IMO the remote repos are for user connivence *ONLY*,

That is horse shit. What they contain is a product of what you people  
put in them. You guys for example have something that is tagged,  
which you released and which is not reproducible. That's not Maven's  
fault, that's your fault. You are hosing anyone trying to consume  
your stuff. And if people are doing that to you then your should  
irate like anyone should be who looks at your tag for 1.1.1. That's  
why the release plugin behaves the way it does. It screamed at you  
when you tried to release that 1.1.1 tag I bet and you side stepped  
what you should have done and did something manually.

> so that you don't need to always go and build everything.  For  
> automated builds it is much easier to build projects one by one,  
> hold on to the exact outputs, then use those outputs as  
> dependencies directly to ensure that dependent builds use the right  
> artifacts.

That is just so wrong if you're using Maven. Why do you go have to  
build everything? It's because your dependency chain consists of so  
many transient SNAPSHOTs that you can't possibly rely on anything.  
Start in your specs and release them,  when you have resolved a few  
issues bump the micro number and release again. Eliminate them  
anywhere you can. Releasing is not painful with the release plugin  
when you have your SNAPSHOTs under control.

> if the project is not going to build as one might expect when  
> running `mvn install` from the root.... and if that is because you  
> want to version each of those bits separately and use the half- 
> functional maven release plugin to handle releases... then each of  
> these really needs to be a self-contained project.

You guys seem to want to use the release plugin for releasing things  
with SNAPSHOTs in them. That's not what it was made for and again  
it's a function of you not managing your relationships with your  
dependent projects. A release with SNAPSHOTs is not a release.  
Tagging something that contains a POM with a SNAPSHOT identifier is  
completely useless. I mean look at your 1.1 release:

That's insane. That's a tag! How did you do a release with SNAPSHOTs?  
That's not Maven's fault if you can't built that again if you were  
careless enough to tag something with SNAPSHOTs. You have got to use  
releases of plugins, parent POMs, and dependences more frequently.  
Maven has lots of bugs, sure, but this mess is largely your own  
doing. There are lots of Maven users with way more source code and  
modules then you and don't have nearly the number of problems you do.

> I really feel that the little short-cuts and side-steps around  
> maven problems are going to kill us.  And really, I don't see how  
> any medium to large sized group can effectively use maven to manage  
> releases of their projects...

Lots of projects do it fine. Your dependency chain is fragile which  
is the problem. Your coupling is too high which will not let you do  
releases as you have to, or you are not working with the producers of  
your dependencies to create releases. In trunk right now you have a  
SNAPSHOT for the POM that seems to contain the dependencies for  
everything you need. That's wrong. You need to release those parent  
POMs and the dependencies in them. It is natural to have SNAPSHOTs  
for the project in question, for CI and while you're developing. But  
a SNAPSHOT in a dependency means that you, as a last resort, had to  
use something in flux because it cannot work otherwise. This  
immediately should signal you to contact that project to fix it, and  
release it. And again, because all your dependent projects seem to  
share many of the developers you should be able to do that.

> especially not by following in the footsteps of the mvn community  
> wrt they way it versions its components or assumes that remote  
> repositories always contain permanent artifacts which are always  
> trusted.

We will have to start not deleting anything from the source side. We  
have always left the "match sides" option on for rsync so that people  
could manage their own emergencies if they really needed to pull  
something out. But our policy for things that we have added to  
central is never to delete them. You must release and use releases if  
you want stability. If you are managing your own SNAPSHOT repository  
then you decide what stays or goes. If you want them to remain  
forever then don't delete them. In Maven 1.x we never removed  
SNAPSHOTs from Ibiblio either. The stability of your repositories is  
entirely your doing. If you don't want us to delete from our side  
then you have to tell us. We have made the assumption that the source  
is careful enough not to burn their clients. We can't watch  
everything. And we do have a fundamental problem at Apache where  
infrastructure doesn't seem to know how Maven works and don't really  
care too. We would be "bothering" them as I was told by a ember of  
the infra team the other day (another story). I have a pending  
proposal which asks if Contegix can be an official part of the Apache  
infrastructure and I would like them to manage all release and  
snapshot repositories which would give us consistent policies. Then  
the repositories will be happy and we won't be "bothering"  
infrastructure here.

There are changes that Dan Fabulich is proposing to deal with massive  
degrees of flux which is modeled on the build system he's helped  
create at his workplace which builds of revision numbers and the  
system feeds those back to dependent projects. Within a project that  
essentially removes the notion of a SNAPSHOT. This would still not  
help you with your current setup because you have dependencies not  
within your project that are SNAPSHOTs. At some point I could see  
there being a central messaging system that could feed these revision  
changes around the world to any subscriber if they wanted to lock  
onto a revision but that too would not help you because eventually  
you need to use a release which is documented and the API locked down.

You are trying to work around the fact that your dependency chain is  
so hosed all you can do is cry. I'm looking through what you call  
releases and you're obviously not using the release plugin so your  
nightmares are of your own making so stop blaming Maven for your  
problems when you're using it wrong. It seems like the scenario is  
"Oh shit, it's time to do a release. Damn, why won't the release  
plugin let me release with SNAPSHOTs? Damn, Maven blows. We're going  
to have to do this by hand so we can release this stuff with  
snapshots" and then you start hand tagging stuff  and jilting stuff  
around maybe diddling with the repository? And that gets you into a  
world of trouble. You have to prepare for a release a long time in  
advance. Anyone watching me trying to prepare for releases of plugins  
will see that I've spent a long time releasing all their dependent  
projects and it takes time and is not all that interesting or fun. In  
order to make the licensing changes I wanted I had to modify many   
pieces at once, but the second it all worked I released all the  
pieces, their parent POMs and locked the client projects to those  
released versions. The only fundamental flaw in the release plugin  
right now is that it won't easily release a parent POM. Aside from  
that if you can't use the release plugin then you're doing something  
wrong somewhere.

Building everything from source is an indicator of severe problems.

Seeing tagged releases with SNAPSHOTs in them is an indication of  
severe problems.

Your repository whimsically shifting around is an indicator of severe  

None of these are inherently Maven's fault


> --jason

View raw message