maven-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dennis Lundberg <denn...@apache.org>
Subject Re: [VOTE] Release Apache Maven Model Converter version 2.3
Date Fri, 16 Aug 2013 11:25:00 GMT
On Fri, Aug 16, 2013 at 11:24 AM, sebb <sebbaz@gmail.com> wrote:

> On 16 August 2013 09:32, Dennis Lundberg <dennisl@apache.org> wrote:
> > On Fri, Aug 16, 2013 at 9:52 AM, sebb <sebbaz@gmail.com> wrote:
> >
> >> On 16 August 2013 08:10, Dennis Lundberg <dennisl@apache.org> wrote:
> >> > On Fri, Aug 16, 2013 at 1:20 AM, sebb <sebbaz@gmail.com> wrote:
> >> >
> >> >> On 15 August 2013 20:57, Dennis Lundberg <dennisl@apache.org>
wrote:
> >> >> > On Thu, Aug 15, 2013 at 9:27 PM, sebb <sebbaz@gmail.com>
wrote:
> >> >> >
> >> >> >> On 15 August 2013 14:16, Jason van Zyl <jason@tesla.io>
wrote:
> >> >> >> > What Sebb is doing is perfectly reasonable.
> >> >> >>
> >> >> >
> >> >> > I agree. Checking that the source bundle is correct is good release
> >> >> review
> >> >> > practice.
> >> >> >
> >> >> > Thank you!
> >> >> >>
> >> >> >> > He's trying to assert that everything in the source ball
> actually
> >> >> comes
> >> >> >> from source control and that no errant files have made their
way
> into
> >> >> the
> >> >> >> distribution. Right now we cannot assert that the assembly
plugin
> has
> >> >> not
> >> >> >> wandered outside the checkout and pulled something else in,
or
> that
> >> >> someone
> >> >> >> didn't accidentally put something else in the distribution.
I
> think
> >> this
> >> >> >> unlikely but we can't assert otherwise right now which I believe
> is
> >> >> Sebb's
> >> >> >> point.
> >> >> >>
> >> >> >> It *has* already happened several times that I am aware of.
> >> >> >>
> >> >> >> The last few releases of the War plugin (various RMs &
voters)
> >> >> >> included at least one spurious file.
> >> >> >> So it was not just a one-off packaging - and review - failure.
> >> >> >> [See separate thread(s) for all the details; they are not
germane
> >> here.]
> >> >> >>
> >> >> >> The message is that mistakes happen even in automated processes.
> >> >> >> Which is why independent comparison of input and output is
> valuable.
> >> >> >>
> >> >> >> > If we can embed the revision from which the assembly
was made in
> >> the
> >> >> >> assembly itself (and maybe the build number plugin is doing
this
> >> >> already),
> >> >> >> then a tool can be made to unpack the assembly, checkout the
> revision
> >> >> and
> >> >> >> assert that everything in the source distribution comes from
> source
> >> >> control.
> >> >> >> >
> >> >> >> > If we can also assert that as part of each build all
the license
> >> files
> >> >> >> are intact and headers are in place then I believe we're done
with
> >> >> >> provenance.
> >> >> >> > Licenses are present, all files have valid license headers,
all
> >> files
> >> >> >> present in the source distribution come from source control,
all
> >> >> >> contributions to source control are from bonafide CLA carrying
> Apache
> >> >> >> committers because you don't get access to commit until the
CLA
> is on
> >> >> file.
> >> >> >> >
> >> >> >> > Sebb, reasonably accurate?
> >> >> >>
> >> >> >> Yes.
> >> >> >>
> >> >> >> One other point I made already is that I think the vote e-mail
> needs
> >> >> >> to be transparent to all, not just those on the PMC.
> >> >> >> Links to the output from the release process are obviously
> already in
> >> >> >> the mail; what is missing is the input to the process, e.g..
SCM
> >> >> >> coords.
> >> >> >> Yes, it may be possible to dig out the details from the archive,
> but
> >> >> >> that's not trivial.
> >> >> >>
> >> >> >
> >> >> > I disagree.
> >> >> >
> >> >> > If we focus first on a "normal" release, one that succeeds on
the
> >> first
> >> >> > attempt, without a respin or deleting of tags.
> >> >> > To check provenance you would do this:
> >> >> >
> >> >> > 1. download the source bundle
> >> >> > 2. unpack the source bundle
> >> >> > 3. checkout the corresponding source code from the SCM
> >> >> > 4. compare the two trees
> >> >> >
> >> >> > Right so far?
> >> >> >
> >> >> > What you want, if I understand you correctly, is to have the SCM
> URL
> >> in
> >> >> the
> >> >> > vote email. So that you can give that to your SCM client in step
3.
> >> >>
> >> >> Yes.
> >> >>
> >> >> > With the process we use here at the Maven project, the SCM URL
is
> in
> >> the
> >> >> > pom.xml file that sits in the root directory of the unpacked source
> >> >> bundle.
> >> >> > All you need to do is open the file and copy the URL from there.
I
> >> still
> >> >> > fail to see how that is so much harder than to copy the URL from
an
> >> >> email.
> >> >> >
> >> >> > That is if you don't know the conventions that we use, by way
of
> the
> >> >> > Release Plugin. The tag will always be in the format
> >> >> > ${project.artifactId}-${project.version}
> >> >> >
> >> >>
> >> >> My point is that it should be completely transparent, even to outside
> >> >> reviewers.
> >> >>
> >> >
> >> > I guess that this is the point that we'll have to agree to disagree
> on.
> >> My
> >> > view is that if someone wants to to review a release from the Maven
> >> > project, they'd have to have a basic understanding of how Maven works
> and
> >> > how we do releases in the Maven project. That includes what the
> Release
> >> > Plugin is and how it works.
> >>
> >> Why does the release tool matter?
> >>
> >
> > It is not the tool itself that matters, but rather what the tools does.
> > Since the Release plugin automates a lot of the tasks involved when
> doing a
> > release, it can appear to make things less transparent. Therefor a
> > knowledge about what it does is needed to make it transparent.
>
> That really is not the point.
>
> It's possible to create the same outputs from the same inputs using
> all sorts of methods, manual or automated.
>
> The fact that it is a Maven release should not matter as far as the
> source checking is concerned, except insofar as the staging area may
> be different.
>
> Only the source release archives matter.
>

All of what you say above is true, but that is not what I'm talking about.
My focus is still on the 2.5 bullet in the process of *reviewing* a source
release, not it's creation.

You want the SCM URL in the vote email. I'm arguing that that piece of
information is already available to you in the source bundle, so there is
not point in including it in the vote email. Knowing *what* the Release
Plugin does makes you aware of this fact.


>
> > It's still an ASF release.
> >>
> >
> > Yes, and we must adhere to all the requirements of an ASF release. There
> > are however a lot of different ways to make an ASF compliant release.
> Each
> > PMC decides on a process for that, that works best for them.
>
> That's fine, the tool and process does not matter so long as the
> output is correct.
>
> > It should not matter how the source release artifacts are generated.
> >> The process of checking them is still the same.
> >>
> >> Of course if things go wrong, then it's important to know how the SCM
> >> input was assembled into the source archives in order to fix the
> >> process.
> >> But the release tool is completely irrelevant to the process of
> >> checking the contents of the source archives.
> >>
> >
> > See above.
> >
> >
> >>
> >> > Most people have their own checklist of stuff they do when reviewing a
> >> > release. Would it be a good idea if we aggregate all those points on a
> >> page
> >> > on the Maven web site, under the development section? That would also
> >> serve
> >> > as a guide for "outside reviewers".
> >>
> >> Yes, though that is a separate issue.
> >>
> >
> > Agreed.
> >
> >>
> >> >> > Now, for a respun release thing are trickier. Here it might be
a
> good
> >> >> idea
> >> >> > to include the revision number or hash, or whatever is unique
in
> the
> >> SCM
> >> >> > being used.
> >> >>
> >> >> And how do you know from a vote e-mail that it is respun?
> >> >>
> >> >
> >> > It should always say so in the subject of the vote email. Not sure if
> >> that
> >> > is written down somewhere, but that's how we have always done it. If
> it's
> >> > missing I'll add it to our release process document.
> >> >
> >> >
> >> >>
> >> >> > Even though the code under review will always be under the
> >> >> > latest tag in the above format (at least for Subversion).
> >> >>
> >> >> Until the next respin.
> >> >>
> >> >> If there is a respin, and reviewers are not following the e-mails
> very
> >> >> carefully, it would be quite easy to overlook an updated tag.
> >> >>
> >> >> This is all about making sure that it is really obvious what the
> vote is
> >> >> about.
> >> >>
> >> >
> >> > Yes, that's why I agreed that it's a good idea to add the
> revision/hash
> >> > when doing a respin.
> >> >
> >> >
> >> >>
> >> >> >
> >> >> >> Publishing the SCM coordinates in the mail is trivial to do,
and
> >> shows
> >> >> >> that the input is an important part of the review process.
> >> >> >> Having the information in the mail thread is also useful for
the
> >> >> archives.
> >> >> >>
> >> >> >
> >> >> > As others have said before, we aim to automate the release process
> as
> >> >> much
> >> >> > as possible. Therefor even a seemingly minor addition will be
> >> questioned,
> >> >> > as you have noticed, before it is included in our process.
> >> >> >
> >> >> > Can you explain why the information is useful for the archives?
> I've
> >> seen
> >> >> > you mentioned that before. Isn't that moot once the release is
> >> approved?
> >> >> > The tag will be in Subversion for the forseable future and noone
> will
> >> >> touch
> >> >> > it. What am I missing?
> >> >>
> >> >> Why would a release need to be revisited?
> >> >> Perhaps someone is complaining that one of our releases contains code
> >> >> it should not.
> >> >> If that is the case, it helps to have the evidence of the release
> vote
> >> >> in plain sight.
> >> >>
> >> >
> >> > Sure it helps in those cases. But the evidence in our case is in the
> the
> >> > source bundle itself, which is even better in my opinion. There you
> have
> >> > the tag in the POM file.
> >>
> >> However, the link to the staging repo is temporary.
> >> Once the repo is published, the staging repo is deleted and there is
> >> no direct route to the pom from the vote e-mail.
> >>
> >
> > Once the staging repo is promoted, its content is moved to the real
> > repository. So the source bundle is available there for anyone who wants
> to
> > track down the complete contents of a previous release.
>
> However that is not the canonical location for source releases; source
> releases MUST be released via the ASF mirror system.
>

So, go download it from the mirror system then. I fail to see how that is
relevant to this discussion.


> > Again this is where a bit of knowledge of what the Release Plugin does
> > helps you. A release done with the Release Plugin will always be
> available
> > at
> >
> https://repository.apache.org/content/repositories/releases/${project.groupId}/${project.artifactId}/${project.version}/${project.artifactId}-${project.version}-source-release.zip
> > where project.groupId has had '.' replaced by '/'.
>
> AIUI the name is not controlled by the Release Plugin.
>
> The first part of the name is determined by Maven Convention and ASF
> hosting.
> The -source-release.zip suffix depends on settings outside the Release
> plugin.
>
> Apache Commons sometimes uses the Release Plugin, but our archives are
> called -src.zip and -src.tar.gz
>

Okay, I'll rephrase that:

Again this is where a bit of knowledge of what the Release Plugin does
*during a release from the Apache Maven project* does help you.

Is that clear enough?


>
> >
> >> > Note that I'm not talking about respins here, that's another story.
> >>
> >> >
> >> >>
> >> >> >
> >> >> >>
> >> >> >> > On Aug 15, 2013, at 9:01 AM, Chris Graham <chrisgwarp@gmail.com
> >
> >> >> wrote:
> >> >> >> >
> >> >> >> >>
> >> >> >> >>
> >> >> >> >> Sent from my iPhone
> >> >> >> >>
> >> >> >> >> On 15/08/2013, at 10:05 PM, sebb <sebbaz@gmail.com>
wrote:
> >> >> >> >>
> >> >> >> >>> On 15 August 2013 10:08, Chris Graham <chrisgwarp@gmail.com>
> >> wrote:
> >> >> >> >>>> What sebb does not appear to have understood
or accepted, as
> >> >> Stephen
> >> >> >> has
> >> >> >> >>>> endlessly pointed out, is that we vote on
the source bundle,
> >> not a
> >> >> scm
> >> >> >> >>>> revision, and that, strictly speaking a SCM
is not even
> required
> >> >> >> (however
> >> >> >> >>>> sensible it is to use one).
> >> >> >> >>>>
> >> >> >> >>>> He wants a tree and a revision so that we
can compare between
> >> >> >> releases,
> >> >> >> >>>> where what he should be doing, strictly speaking,
is
> comparing
> >> >> source
> >> >> >> tar
> >> >> >> >>>> balls, as that is what we really are voting
on.
> >> >> >> >>>
> >> >> >> >>> I agree that what is released (and voted on)
are the source
> >> >> tarballs.
> >> >> >> >>> And any such tarballs should be identical (barring
possibly
> >> >> different
> >> >> >> >>> EOL settings for text files).
> >> >> >> >>>
> >> >> >> >>> However, that is only one of the checks that
need to be made.
> >> >> >> >>>
> >> >> >> >>> The PMC also needs to ensure that the files are
being released
> >> under
> >> >> >> >>> the correct license.
> >> >> >> >>
> >> >> >> >> Are not the licenses in the source that is in the
source
> tarball?
> >> >> >> >>
> >> >> >> >> If so, can not the rat plugin or similar be used
to check the
> >> >> >> compliance?
> >> >> >> >>
> >> >> >> >>> I contend that the only practical way to check
the licences
> is to
> >> >> >> >>> compare the source tarball(s) with the files
in SCM.
> >> >> >> >>>
> >> >> >> >>> [The files should only be added to SCM if the
license is OK,
> so
> >> the
> >> >> >> >>> SCM tag acts as a database of validated source
files.]
> >> >> >> >>>
> >> >> >> >>> The SVN revision / Git hash are needed to ensure
uniqueness.
> >> >> >> >>>
> >> >> >> >>>
> >> >> ---------------------------------------------------------------------
> >> >> >> >>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> >> >> >> >>> For additional commands, e-mail: dev-help@maven.apache.org
> >> >> >> >>>
> >> >> >> >>
> >> >> >> >>
> >> ---------------------------------------------------------------------
> >> >> >> >> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> >> >> >> >> For additional commands, e-mail: dev-help@maven.apache.org
> >> >> >> >>
> >> >> >> >
> >> >> >> > Thanks,
> >> >> >> >
> >> >> >> > Jason
> >> >> >> >
> >> >> >> > ----------------------------------------------------------
> >> >> >> > Jason van Zyl
> >> >> >> > Founder,  Apache Maven
> >> >> >> > http://twitter.com/jvanzyl
> >> >> >> > ---------------------------------------------------------
> >> >> >> >
> >> >> >> > There's no sense in being precise when you don't even
know what
> >> you're
> >> >> >> talking about.
> >> >> >> >
> >> >> >> >  -- John von Neumann
> >> >> >> >
> >> >> >> >
> >> >> >> >
> >> >> >> >
> >> >> >> >
> >> >> >> >
> >> >> >>
> >> >> >>
> ---------------------------------------------------------------------
> >> >> >> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> >> >> >> For additional commands, e-mail: dev-help@maven.apache.org
> >> >> >>
> >> >> >> --
> >> >> >> Dennis Lundberg <dev-help@maven.apache.org>
> >> >> >>
> >> >>
> >> >> ---------------------------------------------------------------------
> >> >> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> >> >> For additional commands, e-mail: dev-help@maven.apache.org
> >> >>
> >> >> --
> >> >> Dennis Lundberg <dev-help@maven.apache.org>
> >> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> >> For additional commands, e-mail: dev-help@maven.apache.org
> >>
> >> --
> >> Dennis Lundberg <dev-help@maven.apache.org>
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
> --
> Dennis Lundberg <dev-help@maven.apache.org>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message