incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jim Apple <jbap...@cloudera.com>
Subject Re: Airflow voting on release artifacts
Date Tue, 25 Apr 2017 22:04:22 GMT
Would virtualenv help by avoiding the "Requirement already satisfied"
complaints?

On Tue, Apr 25, 2017 at 2:05 PM, Bolke de Bruin <bdbruin@gmail.com> wrote:

> Thank you thinking with us, that’s really appreciated. Unfortunately, many
> of the assumptions from the Java world do not apply to the Python world:
>
> Version information inside the artefact needs to be in sync with the
> external filename.
>
> Examples:
>
> 1. Plain rename of the tar ball
>
> bolke$ pip install airflow-1.8.0rc11+apache.incubating.tar.gz
> Processing ./airflow-1.8.0rc11+apache.incubating.tar.gz
>   Requirement already satisfied (use --upgrade to upgrade):
> airflow==1.8.0rc5+apache.incubating from file:///Users/bolke/Documents/
> dev/airflow/dist/airflow-1.8.0rc11%2Bapache.incubating.tar.gz in
> /Users/bolke/Documents/dev/airflow_env/lib/python2.7/site-packages
>
> Note that although RC11 is mentioned RC5 is contained in the package and
> is not upgraded.
>
> 2. Rename of the package inside the tar ball
> bolke$ mv airflow-1.8.0rc5+apache.incubating airflow-1.8.0rc11+apache.
> incubating
> bolke$ tar cfz ../airflow-1.8.0rc11+apache.incubating.tar.gz
> airflow-1.8.0rc11+apache.incubating/
>
> bolke$ pip install airflow-1.8.0rc11+apache.incubating.tar.gz
> Processing ./airflow-1.8.0rc11+apache.incubating.tar.gz
>   Requirement already satisfied (use --upgrade to upgrade):
> airflow==1.8.0rc5+apache.incubating from file:///Users/bolke/Documents/
> dev/airflow/dist/airflow-1.8.0rc11%2Bapache.incubating.tar.gz in
> /Users/bolke/Documents/dev/airflow_env/lib/python2.7/site-packages
>
> 3. There is no “separate” build script. Pip will just install a binary
> (“wheel”) or uses the source package (as shown above). Both are used
> interchangeable by users. We only distribute source packages at the moment.
>
> @Alex: I have to think a little bit more about what you wrote, but it is
> currently confusing the hell out of me :). Furthermore, I am not sure if it
> applies considering the above #3.
>
> At the moment we seem stuck between a rock and a hard place. And as we
> would like to release a new version pretty soon, we probably want to vote
> twice for now, and including the IPMC, four times. Hopefully we can have a
> vote in rapid succession :-).
>
> Bolke
>
>
> > On 25 Apr 2017, at 17:33, Alex Harui <aharui@adobe.com> wrote:
> >
> >
> >
> > On 4/25/17, 1:43 AM, "Bolke de Bruin" <bdbruin@gmail.com <mailto:
> bdbruin@gmail.com>> wrote:
> >
> >> Hi Bertrand,
> >>
> >>> On 25 Apr 2017, at 09:04, Bertrand Delacretaz
> >>> <bdelacretaz@codeconsult.ch> wrote:
> >>>
> >>> Hi,
> >>>
> >>> On Mon, Apr 24, 2017 at 10:18 PM, Chris Riccomini
> >>> <criccomini@apache.org> wrote:
> >>>> ...Hitesh recently raised the issue that the artifact that passes the
> >>>> vote
> >>>> MUST be the one that we actually release...
> >>>
> >>> Yes in terms of having the same binary digests and signatures, but
> >>> renaming the files is fine IMO, especially for removing an -rc suffix
> >>> which makes total sense. I would just add that step to your release
> >>> process documentation to make it clear.
> >>>
> >>>> ...Rename/rebuild after final vote (This is what Airflow is doing, and
> >>>> Beam
> >>>> does this, I believe)...
> >>>
> >>> I'd say rename yes but rebuild no, in order to keep the same digests
> >>> and signatures.
> >>>
> >>
> >> As mentioned earlier, that seems not to be possible. The metadata
> >> (filename) and version information inside the package need to be in
> sync.
> >> This how the python build tools and python ecosystem works.
> >
> > I'm not familiar with Python, but is it possible to have a command line
> > option that adds the "-rc" suffix in the right places?
> >
> > IMO, there are two "audiences".  One is the voters, and one is your
> > customers.  The customers should not be using the RC until after it is
> > approved unless they are volunteering to be a voter.  Voters are
> primarily
> > supposed to examine a source artifact, but if you also release a
> > "convenience binary" artifact, there are some examinations of that
> > "binary" artifact required.  For many projects this "convenience"
> artifact
> > is the one that the vast majority of customers consume.  AIUI, Python
> > doesn't have binaries, but IMO, there is no requirement that this
> > "convenience" artifact actually contains binaries.  A "convenience"
> > artifact is just supposed to be the source artifact processed by a build
> > script since many of your customer may not have, or may not want to
> > configure their machines to run whatever build script engine you choose
> > for your project.  Further, there is no requirement that I know of that
> > voters must test the "convenience" artifact by actually running it.  It
> > just makes sense to do so in most cases.  But if Python is interpreted
> > source, you may be able to use this to your advantage.
> >
> > So, your source package should consist of the source and build script
> > required to build this "customer"/"convenience" package, and the build
> > script should allow a command line option to add the "-rc" suffix.  Then
> > voters would be instructed to download the source package, and to test
> it,
> > build a "customer" package with the "-rc" suffix and test the results.
> > And voters would also be instructed to download the "customer" package
> > that doesn't have the -rc suffix.  But in order to test it, they only
> have
> > to diff that package against the "customer" package they built from the
> > source package.  It should be the same except for the metadata.
> >
> > We'll see if others can find a problem with this plan, but IMO, that
> would
> > be good enough for me as an PMC voter.
> >
> > HTH,
> > -Alex
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> <mailto:general-unsubscribe@incubator.apache.org>
> > For additional commands, e-mail: general-help@incubator.apache.org
> <mailto:general-help@incubator.apache.org>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message