maven-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Curtis Rueden <ctrue...@wisc.edu>
Subject Re: Jar file not in maven
Date Wed, 30 Jan 2013 17:14:50 GMT
Hi everyone,

First of all, thanks for the great discussion, everyone!

Stephen: I really appreciate you taking the time to lay out your solution
with download-single. That is quite slick!

Joachim: Thanks for being tenacious about getting your questions answered
with a real technical discussion.

A few comments from me:

1) I absolutely agree that this list exists for technical discussions. If
people are too tired of debating the same topics many times, linking to a
FAQ is by far the best solution. Taking the time to write such FAQs goes a
very long way toward alleviating repeat questions with complicated answers.

2) Stephen's makes a lot of good points, but one in particular is really
paramount: you *must* define the metadata for each artifact: groupId,
artifactId, version, and dependencies. If you are looking for an automated
& repeatable solution for linking to an external (i.e., non-Mavenized)
dependency online somewhere -- even taking as a given that the URL is
stable & immutable -- there is still the issue of its GAV and dependencies.
Those must be defined somewhere. The most effective approach is to deploy
the JAR + POM to a Maven repository somewhere, which is one reason people
keep harping on MRMs.

3) That said, there are many ways of skinning that cat, as Stephen says.

If you have a server you control -- and which all involved project
developers can access -- you can use an MRM to upload the JAR + POM
metadata once and for all, as Ron says. You do it once, and it is done
forever. Builds are reliable and repeatable. Provenance is preserved inside
the POM (more on that below).

If you do not have a server you control, or you think running an MRM is too
complex, or some other issue, there is install:install-file. Now, I
absolutely appreciate the concern that it sucks to force all your
developers to do that on every new local machine. But there are ways around
it. One simple way is to commit a bootstrapping script to your SCM that
automates the entire process:
  - Create the POMs for your non-Mavenized projects (as discussed above in
point #2).
  - Commit these POMs to your SCM.
  - Write a script that A) downloads the JARs from the external stable
URLs; and B) calls "mvn install:install-file" with those JARs +
corresponding POMs.
  - Commit that script to your SCM.

Now all your developers have to do is run the script after cloning your
repo, and they are good to go: repeatable builds for as long as they have a
~/.m2/repository on disk.

If you are not 100% absolutely sure the external URLs are stable and
immutable, you can commit the JARs themselves to your SCM instead of
letting the script download them. (And feel the nostalgia of being back to
the good ol' days of pure Ant!)

If your non-Mavenized projects have no dependencies, you can even skip the
manual POM creation and let Maven generate a "crappy stub" as Stephen says.
But I think it is better to define the POMs anyway because you can write
all sorts of metadata goodness in there (see next paragraph).

Regarding Joachim's concern of preserving the metadata: that is one of
Maven's great strengths. In the POMs you define, you can add all the
information you want. You can define the developers & contributors to the
project. You can define the project's web site URL. You can define the SCM
used. You can define the issue tracker used. The list goes on and on. [1, 2]

Personally, I do run an MRM for my projects (Nexus on a public server we
control) because it is useful for my team and for the OSS community -- no
need for developers to run a bootstrapping script even once -- things "just
work" when you run "mvn" with no arguments. Nonetheless, I still have my
third-party non-Mavenized POMs committed to an SCM too, just to remind
myself and others what they are.

OK, so after reading all that, you might think it could be nice to have a
Maven plugin that does what the shell script above would do, given a
directory full of POMs or something. Then you could bind it to a phase of
the build and things would "just work" again without developers needing to
manually bootstrap. So lastly, I agree with Wayne that if you want
something like that (because you can't run an MRM for some reason), feel
free to scratch your itch. It would be a valuable addition to the Maven
ecosystem!

Regards,
Curtis

[1] Maven POM reference lists it all: http://maven.apache.org/pom.html
[2] An example of a rich POM:
https://github.com/imagej/imagej/blob/master/pom.xml


On Wed, Jan 30, 2013 at 3:44 AM, Stephen Connolly <
stephen.alan.connolly@gmail.com> wrote:

> On 30 January 2013 09:15, Joachim Durchholz <jo@durchholz.org> wrote:
>
> > Am 30.01.2013 09:13, schrieb Anders Hammar:
> >
> >  Joachim, a possible solution would be to front the external non-Maven
> repo
> >> with a repo manager transforming it into a true Maven repo.
> >>
> >
> > Sounds complicated.
> > I.e. not doable unless you know all aspects that need to be considered,
> > hence probably far out of my league.
> >
> >
> > > As a Nexus (one
> >
> >> of the repo managers out there) user I know you can do that via
> >> implementing a virtual repo plugin, but I would expect you can do
> similar
> >> things with the other repo managers as well.
> >>
> >
> > Still too complicated and out of my league, I fear.
> >
> >
> > > Just keep in mind that it
> >
> >> can't do anything magical; if metadata is missing you have to provide it
> >> in
> >> some way.
> >>
> >
> > Sure.
> > The version number would come from the revision number or tag if using an
> > SVN as a source.
> > For Sourceforge, it would have to be extracted from the file name.
> >
> > I think the version number is the most important metadata anyway.
> >
> >
> >  Other than that, I can only shim in with Stephen that many of us are
> tired
> >> of arguing about what's the correct way of solving things "the Maven
> way".
> >>
> >
> > Yes, I can understand that. I'm pretty much aware that this is making
> this
> > exchange tortuous, for both sides; I have to constantly battle this "you
> > essentially don't know what you're talking about" attitude, which is
> tiring
> > as well.
> >
> >
> >  You may not agree with us, but that's a different thing.
> >>
> >
> > Yes, currently I disagree.
> >
> > However, I'm eager to hear arguments that may falsify my view. I'd be all
> > for it to see how this use case (import available from an external,
> stable
> > download site) can be properly integrated into a Maven workflow.
> >
> > The proposals that I have seen don't preserve the version number. My
> claim
> > (yet to be falsified) is that this makes people set up imports without
> > version number, losing this essential piece of information. If people "do
> > it right" and manually add the version number by whatever means, a
> consumer
> > of such a Maven artifact still won't know whether the manually provided
> > information is actually right.
> > So I'm arguing for some mechanism to record in a pom:
> > - the coordinate where the download comes from
> > - the coordinate where the artifact is supposed to go do in a maven repo
> > - an automatic process that installs the pom and the download
> > Whether the download should be part of the build process or triggered
> > manually depends on whether the origin is stable (i.e. whether there's a
> > risk that the origin overwrites the artifact at its coordinate - not an
> > issue if the origin is an SCM and the download coordinate includes a
> > revision number).
> >
>
> Here is how I would "Maven Way" handle this kind of dependency.
>
> First off, I am assuming (always a danger of making an ass of you and me)
> that we have a multi-module project.
>
> I would create a pom.xml for the artifact that I want to use. This pom.xml
> will have to correctly identify the runtime dependencies of the artifact I
> am providing, so tooling is not really going to help, the artifact will
> have to be hand written due to the nature of these things.
>
> [I have experimented with using bytecode analysis to establish dependencies
> between a set of .jar files in a lib directory, and it can give you a good
> starting point, but too often it gives dependencies which should be
> optional, or misses dependencies that are referenced via reflection. The
> amount of times it has been in error is > 20% in my experience, and that
> error rate is unacceptable for general guidance]
>
> So you have to write the pom.xml by hand.
>
> It will look something like this
>
> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="
> http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="
> http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd
> ">
>   <modelVersion>4.0.0</modelVersion>
>   <groupId>com.yourcompany.thirdparty</groupId>
>   <artifactId>third-party-artifact</artifactId>
>   <version>subversion-revision</version>
>
>   ... other details as you see fit ...
>
>   <dependencies>
>     ... list these ...
>   </dependencies>
>
>   <build>
>     <plugins>
>       <!-- many ways to skin this cat -->
>       <plugin>
>         <groupId>org.codehaus.mojo</groupId>
>         <artifactId>wagon-maven-plugin</artifactId>
>         <version>1.0-beta-4</version>
>         <executions>
>           <execution>
>             <phase>package</phase>
>             <goals>
>               <goal>download-single</goal>
>             </goals>
>             <configuration>
>               <url>some remote url</url>
>               <fromFile>third-party-artifact.jar@
> ${project.version}</fromFile>
>
>
> <toFile>${project.build.outputDirectory}/${project.build.finalName}.jar</toFile>
>             </configuration>
>           </execution>
>         </executions>
>       </plugin>
>     </plugins>
>   </build>
>
> </project>
>
> Now that pom.xml implicitly defines the location of the artifact, though
> you need to read the pom to find out. Your project will be a simple
> "check-out and build" assuming they are not behind a corporate proxy that
> prevents downloading .jar files from random websites... in which case they
> will need to get an exception for all of them to access that website and
> they will curse you for not just sticking it in a MRM which they at least
> got the corporate IT guys to grant an exception to...
>
> The better way would be to write the pom.xml and use
>
> mvn install:install-file -Dfile=... -DpomFile=...
>
> or
>
> mvn deploy:deploy-file -Dfile=... -DpomFile=...
>
> or upload as a bundle to the corporate MRM
>
> to install that pom... the pom would probably look mostly the same, except
> you would list the origin of the file in the <scm> section, e.g.
>
> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="
> http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="
> http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd
> ">
>   <modelVersion>4.0.0</modelVersion>
>   <groupId>com.yourcompany.thirdparty</groupId>
>   <artifactId>third-party-artifact</artifactId>
>   <version>subversion-revision</version>
>
>   <scm>
>     <connection>http://some-remote-url/...</connection>
>   </connection>
>
>   ... other details as you see fit ...
>
>   <dependencies>
>     ... list these ...
>   </dependencies>
>
> </project>
>
> And here's the best bit about that way... when you write a pom.xml like
> that, it's just perfect for submitting to central in a bundle upload...
> which means that others can receive the benefit of your hard work
> determining the list of dependencies.
>
> Now if you don't provide a pom file to install:install-file or
> deploy:deploy-file those plugins will generate a crappy stub for you... we
> need to put some pom there, and the stub will not break things, so it
> should be seen as a stop-gap until you determine the correct <dependencies>
>
> Now if you are complaining that the above is too much work, well the only
> real work I see is determining the list of dependencies... and sorry to
> say, but that is hard work that a person needs to do if it is to be done
> right... and if you don't need it done right, then the stub pom is just
> fine!
>
>
> >
> > ------------------------------**------------------------------**---------
> > To unsubscribe, e-mail: users-unsubscribe@maven.**apache.org<
> users-unsubscribe@maven.apache.org>
> >
> > For additional commands, e-mail: users-help@maven.apache.org
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message