commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stian Soiland-Reyes <st...@apache.org>
Subject Re: [ALL] Dist layout change to per version directories
Date Tue, 03 May 2016 21:11:30 GMT
Because e.g. Math 2.2 was released on 2013-03-03 - a "re-release" of
2.2 at a new location is unexpected - at least if the old location
goes 404.


You are right that my approach would not make 2.1 available in the new
layout - which I think there is not much point in as naturally there
are other inconsistencies across components and filenames for any code
historian browsing https://archive.apache.org/dist/commons/

The goal of my approach was just ensuring that the latest version of
every commons component is available in the new layout (consistency,
as raised by Gilles) - without breaking the active download links on
the mirrors for existing versions that have not (yet or ever) been
replaced.

Practically using symlinks on dist.apache.org in SVN is probably an
easier change to implement as you don't need to re-upload 500 MB or
script thousands of svn cp commands (and check they all worked).


Just my take - I'm not insisting on this - go ahead and break the web
:)  At least then the new layout will be picked up quickly..

On 3 May 2016 at 16:18, sebb <sebbaz@gmail.com> wrote:
> On 3 May 2016 at 11:13, Stian Soiland-Reyes <stain@apache.org> wrote:
>> Trying to summarize here.. :)
>>
>> On 25 April 2016 at 10:49, sebb <sebbaz@gmail.com> wrote:
>>
>>>>> Does it really matter if the URL changes more than just a version string?
>>>> Yes,
>>> I don't understand why that should be.
>>> Can you explain in more detail?
>>
>> Mainly in download recipes with a URL pattern patterns using
>> ${version} or similar.
>>
>> For instance:
>>
>> https://anonscm.debian.org/cgit/pkg-java/commons-io.git/tree/debian/watch
>> https://git.archlinux.org/svntogit/community.git/tree/trunk/PKGBUILD?h=packages/java-commons-io
>>
>>
>> Obviously these patterns will for the next versions need to change
>> anyway with the new layout (they already have to deal with the fact
>> that only the newest version is in the mirrors, that's why Debian
>> clone into their own git repositories based on that watch file)
>>
>> Personally when using Apache software from a Docker build file I have
>> to use http://archive.apache.org/ fallback URLs to make sure my recipe
>> keeps working - see for instance
>> https://github.com/stain/jena-docker/blob/master/jena/Dockerfile#L34
>>
>> But the old releases won't be affected on archive.apache.org (e.g.
>> they will stay in */source */binarines).
>>
>>
>>
>>> Unless we continue providing links for every future release, the
>>> downstream provider has to change the download URL.
>>> Once they have done that, why should there be a need to refer to
>>> previous releases at all?
>>> And if there is a need to do so, does it really matter if the URL is different?
>>
>> I agree it doesn't matter if it's a new versions - as they will come
>> with release emails and so on. However I don't think consumers expect
>> existing Download links to change over night without a corresponding
>> new release email for that component.
>>
>> We should of course do a general release email about the download link
>> changes - but then we still have a job to ensure we have updated all
>> our own download pages correctly :)
>>
>> https://archive.apache.org/dist/commons/ would still keep the older
>> versions under binaries/ and sources/ - I don't suggest to make
>> symlinks for every old release in there to the new pattern.
>>
>>
>> The question is what to do with the existing releases in the mirrors.
>>
>> I think from INFRA-11702 we have OK from INFRA to use symlinks for
>> this if we want - e.g. from the new folder structure to the old one -
>> so my suggestion is that we make the change smoother for downstream
>> like that.
>>
>> e.g let's consider
>>
>> https://dist.apache.org/repos/dist/release/commons/math/
>>
>> Current structure (for simplicity only showing *.tar.gz):
>>
>> ./source/commons-math-2.2-src.tar.gz
>> ./source/commons-math3-3.6-src.tar.gz
>> ./source/commons-math3-3.6.1-src.tar.gz
>> ./source/commons-math3-3.5-src.tar.gz
>> ./binaries/commons-math3-3.5-bin.tar.gz
>> ./binaries/commons-math-2.2.tar.gz
>> ./binaries/commons-math3-3.6-bin.tar.gz
>> ./binaries/commons-math3-3.6.1-bin.tar.gz
>>
>>
>>
>> We make the new version folders as you suggested (vary for your
>> preferred 3.5 / math3-3.5 vs math-3.5 folder name style)
>>
>> Adding folders:
>> 2.2/
>> 3.5/
>> 3.6/
>> 3.6.1/
>>
>> - and then (as we've not made new math releases yet) their files
>> symlink back, e.g.:
>>
>> stain@biggie:/tmp/math$ ls -al [23]*/*gz
>> lrwxrwxrwx 1 stain stain 37 May  3 10:41
>> 2.2/commons-math-2.2-src.tar.gz ->
>> ../source/commons-math-2.2-src.tar.gz
>> lrwxrwxrwx 1 stain stain 35 May  3 10:41 2.2/commons-math-2.2.tar.gz
>> -> ../binaries/commons-math-2.2.tar.gz
>> lrwxrwxrwx 1 stain stain 40 May  3 10:43
>> 3.5/commons-math3-3.5-bin.tar.gz ->
>> ../binaries/commons-math3-3.5-bin.tar.gz
>> lrwxrwxrwx 1 stain stain 38 May  3 10:42
>> 3.5/commons-math3-3.5-src.tar.gz ->
>> ../source/commons-math3-3.5-src.tar.gz
>> lrwxrwxrwx 1 stain stain 40 May  3 10:43
>> 3.6/commons-math3-3.6-bin.tar.gz ->
>> ../binaries/commons-math3-3.6-bin.tar.gz
>> lrwxrwxrwx 1 stain stain 38 May  3 10:43
>> 3.6/commons-math3-3.6-src.tar.gz ->
>> ../source/commons-math3-3.6-src.tar.gz
>> lrwxrwxrwx 1 stain stain 42 May  3 11:04
>> 3.6.1/commons-math3-3.6.1-bin.tar.gz ->
>> ../binaries/commons-math3-3.6.1-bin.tar.gz
>> lrwxrwxrwx 1 stain stain 40 May  3 11:04
>> 3.6.1/commons-math3-3.6.1-src.tar.gz ->
>> ../source/commons-math3-3.6.1-src.tar.gz
>>
>>
>> (Math is a special example that has 4 current versions - for say
>> https://dist.apache.org/repos/dist/release/commons/io/ there would
>> only be a single folder,
>> 2.5)
>>
>>
>>
>> Thus the new folders look like this:
>>
>> stain@biggie:/tmp/math/3.6$ ls -al
>> total 0
>> drwxrwxr-x 2 stain stain 360 May  3 10:43 .
>> drwxrwxr-x 8 stain stain 220 May  3 10:42 ..
>> lrwxrwxrwx 1 stain stain  40 May  3 10:43 commons-math3-3.6-bin.tar.gz
>> -> ../binaries/commons-math3-3.6-bin.tar.gz
>> lrwxrwxrwx 1 stain stain  44 May  3 10:43
>> commons-math3-3.6-bin.tar.gz.asc ->
>> ../binaries/commons-math3-3.6-bin.tar.gz.asc
>> lrwxrwxrwx 1 stain stain  44 May  3 10:43
>> commons-math3-3.6-bin.tar.gz.md5 ->
>> ../binaries/commons-math3-3.6-bin.tar.gz.md5
>> lrwxrwxrwx 1 stain stain  45 May  3 10:43
>> commons-math3-3.6-bin.tar.gz.sha1 ->
>> ../binaries/commons-math3-3.6-bin.tar.gz.sha1
>> lrwxrwxrwx 1 stain stain  37 May  3 10:43 commons-math3-3.6-bin.zip ->
>> ../binaries/commons-math3-3.6-bin.zip
>> lrwxrwxrwx 1 stain stain  41 May  3 10:43
>> commons-math3-3.6-bin.zip.asc ->
>> ../binaries/commons-math3-3.6-bin.zip.asc
>> lrwxrwxrwx 1 stain stain  41 May  3 10:43
>> commons-math3-3.6-bin.zip.md5 ->
>> ../binaries/commons-math3-3.6-bin.zip.md5
>> lrwxrwxrwx 1 stain stain  42 May  3 10:43
>> commons-math3-3.6-bin.zip.sha1 ->
>> ../binaries/commons-math3-3.6-bin.zip.sha1
>> lrwxrwxrwx 1 stain stain  38 May  3 10:43 commons-math3-3.6-src.tar.gz
>> -> ../source/commons-math3-3.6-src.tar.gz
>> lrwxrwxrwx 1 stain stain  42 May  3 10:43
>> commons-math3-3.6-src.tar.gz.asc ->
>> ../source/commons-math3-3.6-src.tar.gz.asc
>> lrwxrwxrwx 1 stain stain  42 May  3 10:43
>> commons-math3-3.6-src.tar.gz.md5 ->
>> ../source/commons-math3-3.6-src.tar.gz.md5
>> lrwxrwxrwx 1 stain stain  43 May  3 10:43
>> commons-math3-3.6-src.tar.gz.sha1 ->
>> ../source/commons-math3-3.6-src.tar.gz.sha1
>> lrwxrwxrwx 1 stain stain  35 May  3 10:43 commons-math3-3.6-src.zip ->
>> ../source/commons-math3-3.6-src.zip
>> lrwxrwxrwx 1 stain stain  39 May  3 10:43
>> commons-math3-3.6-src.zip.asc ->
>> ../source/commons-math3-3.6-src.zip.asc
>> lrwxrwxrwx 1 stain stain  39 May  3 10:43
>> commons-math3-3.6-src.zip.md5 ->
>> ../source/commons-math3-3.6-src.zip.md5
>> lrwxrwxrwx 1 stain stain  40 May  3 10:43
>> commons-math3-3.6-src.zip.sha1 ->
>> ../source/commons-math3-3.6-src.zip.sha1
>>
>>
>> e.g. the new-style download link pattern would become
>>
>> http://www.eu.apache.org/dist/commons/math/3.6/commons-math3-3.6-src.tar.gz
>> http://www.eu.apache.org/dist/commons/math/3.6/commons-math3-3.6-bin.tar.gz
>>
>> We then update download pages/templates to use this pattern.
>>
>>
>> For a new release (e.g. commons-math-2.3 or commons-math3-3.7) the
>> files would be added directly to their version folders 2.3/ and 3.7/ -
>> e.g. they would not exist in binaries/ and source) - when clearing a
>> old folder io/2.2/ we would then also clear binaries/*2.2* and
>> sources/*2.2*
>
> That is where I have a problem.
> If it's OK for MATH 2.3 to only exist in the new layout, why is it not
> OK for the current release?
>
> With the above scenario, only MATH 2.2 would be available in both layouts.
> MATH 2.1 and earlier would be old only and MATH 2.3+ new only.
>
> I just don't see how it helps to have a single MATH2 version available
> in both layouts.
>
> Note that there are quite a few Commons components that are unlikely
> to see another release, so providing new-style links for them is a
> waste of time and space.
>
>>
>> Complete example:
>>
>> https://paste.apache.org/8MKJ
>>
>>
>> (Note for this example commons-math-2.2  includes
>> commons-math-2.2-src.tar.gz and the binary commons-math-2.2.tar.gz
>> as we agreed not to rename to -bin on older)
>>
>>
>>
>>
>>
>> --
>> Stian Soiland-Reyes
>> Apache Taverna (incubating), Apache Commons RDF (incubating)
>> http://orcid.org/0000-0001-9842-9718
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>



-- 
Stian Soiland-Reyes
Apache Taverna (incubating), Apache Commons RDF (incubating)
http://orcid.org/0000-0001-9842-9718

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org


Mime
View raw message