incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bolke de Bruin <bdbr...@gmail.com>
Subject Re: Airflow voting on release artifacts
Date Tue, 25 Apr 2017 21:05:30 GMT
Thank you thinking with us, that’s really appreciated. Unfortunately, many of the assumptions
from the Java world do not apply to the Python world:

Version information inside the artefact needs to be in sync with the external filename. 

Examples:

1. Plain rename of the tar ball

bolke$ pip install airflow-1.8.0rc11+apache.incubating.tar.gz
Processing ./airflow-1.8.0rc11+apache.incubating.tar.gz
  Requirement already satisfied (use --upgrade to upgrade): airflow==1.8.0rc5+apache.incubating
from file:///Users/bolke/Documents/dev/airflow/dist/airflow-1.8.0rc11%2Bapache.incubating.tar.gz
in /Users/bolke/Documents/dev/airflow_env/lib/python2.7/site-packages

Note that although RC11 is mentioned RC5 is contained in the package and is not upgraded.

2. Rename of the package inside the tar ball
bolke$ mv airflow-1.8.0rc5+apache.incubating airflow-1.8.0rc11+apache.incubating
bolke$ tar cfz ../airflow-1.8.0rc11+apache.incubating.tar.gz airflow-1.8.0rc11+apache.incubating/

bolke$ pip install airflow-1.8.0rc11+apache.incubating.tar.gz
Processing ./airflow-1.8.0rc11+apache.incubating.tar.gz
  Requirement already satisfied (use --upgrade to upgrade): airflow==1.8.0rc5+apache.incubating
from file:///Users/bolke/Documents/dev/airflow/dist/airflow-1.8.0rc11%2Bapache.incubating.tar.gz
in /Users/bolke/Documents/dev/airflow_env/lib/python2.7/site-packages

3. There is no “separate” build script. Pip will just install a binary (“wheel”) or
uses the source package (as shown above). Both are used interchangeable by users. We only
distribute source packages at the moment.

@Alex: I have to think a little bit more about what you wrote, but it is currently confusing
the hell out of me :). Furthermore, I am not sure if it applies considering the above #3.

At the moment we seem stuck between a rock and a hard place. And as we would like to release
a new version pretty soon, we probably want to vote twice for now, and including the IPMC,
four times. Hopefully we can have a vote in rapid succession :-).

Bolke


> On 25 Apr 2017, at 17:33, Alex Harui <aharui@adobe.com> wrote:
> 
> 
> 
> On 4/25/17, 1:43 AM, "Bolke de Bruin" <bdbruin@gmail.com <mailto:bdbruin@gmail.com>>
wrote:
> 
>> Hi Bertrand,
>> 
>>> On 25 Apr 2017, at 09:04, Bertrand Delacretaz
>>> <bdelacretaz@codeconsult.ch> wrote:
>>> 
>>> Hi,
>>> 
>>> On Mon, Apr 24, 2017 at 10:18 PM, Chris Riccomini
>>> <criccomini@apache.org> wrote:
>>>> ...Hitesh recently raised the issue that the artifact that passes the
>>>> vote
>>>> MUST be the one that we actually release...
>>> 
>>> Yes in terms of having the same binary digests and signatures, but
>>> renaming the files is fine IMO, especially for removing an -rc suffix
>>> which makes total sense. I would just add that step to your release
>>> process documentation to make it clear.
>>> 
>>>> ...Rename/rebuild after final vote (This is what Airflow is doing, and
>>>> Beam
>>>> does this, I believe)...
>>> 
>>> I'd say rename yes but rebuild no, in order to keep the same digests
>>> and signatures.
>>> 
>> 
>> As mentioned earlier, that seems not to be possible. The metadata
>> (filename) and version information inside the package need to be in sync.
>> This how the python build tools and python ecosystem works.
> 
> I'm not familiar with Python, but is it possible to have a command line
> option that adds the "-rc" suffix in the right places?
> 
> IMO, there are two "audiences".  One is the voters, and one is your
> customers.  The customers should not be using the RC until after it is
> approved unless they are volunteering to be a voter.  Voters are primarily
> supposed to examine a source artifact, but if you also release a
> "convenience binary" artifact, there are some examinations of that
> "binary" artifact required.  For many projects this "convenience" artifact
> is the one that the vast majority of customers consume.  AIUI, Python
> doesn't have binaries, but IMO, there is no requirement that this
> "convenience" artifact actually contains binaries.  A "convenience"
> artifact is just supposed to be the source artifact processed by a build
> script since many of your customer may not have, or may not want to
> configure their machines to run whatever build script engine you choose
> for your project.  Further, there is no requirement that I know of that
> voters must test the "convenience" artifact by actually running it.  It
> just makes sense to do so in most cases.  But if Python is interpreted
> source, you may be able to use this to your advantage.
> 
> So, your source package should consist of the source and build script
> required to build this "customer"/"convenience" package, and the build
> script should allow a command line option to add the "-rc" suffix.  Then
> voters would be instructed to download the source package, and to test it,
> build a "customer" package with the "-rc" suffix and test the results.
> And voters would also be instructed to download the "customer" package
> that doesn't have the -rc suffix.  But in order to test it, they only have
> to diff that package against the "customer" package they built from the
> source package.  It should be the same except for the metadata.
> 
> We'll see if others can find a problem with this plan, but IMO, that would
> be good enough for me as an PMC voter.
> 
> HTH,
> -Alex
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org <mailto:general-unsubscribe@incubator.apache.org>
> For additional commands, e-mail: general-help@incubator.apache.org <mailto:general-help@incubator.apache.org>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message