Mailing-List: contact dev-help@airflow.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@airflow.incubator.apache.org
MIME-Version: 1.0
In-Reply-To: <387F5D76-2AA7-44B6-BF0E-DC1C71F64DC9@gmail.com>
References: <CAHEEp7VmK1aBbnTtocaFjHTL3tUVhjt=atRaRUt7K7_DieZBPg@mail.gmail.com>
 <B93DFE58-8682-45B9-B0BF-CA12C6A630EC@gmail.com> <CAHEEp7XF6-b6QjHNo==t1mavS9bNB95eKjSUfgV9DjmmWEpkrg@mail.gmail.com>
 <CABYbY7cME6AYZp=QCu1Ma-tV1RhPXXBoyYTKzMDhiAF4mU8Lbw@mail.gmail.com>
 <CAOgkC2O0Wdo2JLaa_V7sw25OTwpEqEAcYjKym6G1PskiWj1e-Q@mail.gmail.com>
 <5BCC42D0-39ED-46F4-8091-5A060AEF0B1B@firemirror.com> <CAOgkC2N6i6p6LhG1wpZnt=zdjNgOn7SbVTZkDf131FFrgCxjFQ@mail.gmail.com>
 <CAOgkC2N=YvYo7t3HZJ9JNubTyZfYzktSmBa3_74+5TPzXWFRig@mail.gmail.com> <387F5D76-2AA7-44B6-BF0E-DC1C71F64DC9@gmail.com>
From: Maxime Beauchemin <maximebeauchemin@gmail.com>
Date: Thu, 3 Aug 2017 16:00:34 -0700
Message-ID: <CAHEEp7VR2mgq+BDY2qk0KGndo2SSTU_YKOMG4GmEKNLzYVsyFQ@mail.gmail.com>
Subject: Re: [VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC3
To: dev@airflow.incubator.apache.org
Content-Type: multipart/alternative; boundary="f403045f4fa4272fb70555e1595e"
archived-at: Thu, 03 Aug 2017 23:00:44 -0000

--f403045f4fa4272fb70555e1595e
Content-Type: text/plain; charset="UTF-8"

How ok so I publish 2 tarballs (and their 3 related checksums for each) one
for source and one for the sdist.  How do people know it's RC3? Should I
create a folder under `dist/dev/incubator/airflow/1.8.2rc3/`? How do the
names reflect source versus bin (?), how about these filenames:

apache-airflow-1.8.2+incubating-source.tar.gz
<https://dist.apache.org/repos/dist/dev/incubator/airflow/apache-airflow-1.8.2rc3+incubating.tar.gz>
apache-airflow-1.8.2+incubating-bin.tar.gz

Also to recap:
* I change `version.py` to be reference "1.8.2"
* I push the bin tarball referenced to Pypi upon validation

Am I missing something?

Max

On Thu, Aug 3, 2017 at 1:52 PM, Bolke de Bruin <bdbruin@gmail.com> wrote:

> +1, this would solve A LOT of release struggles.
>
> Thanks @ash
>
> Bolke
>
> Sent from my iPhone
>
> > On 3 Aug 2017, at 22:39, Hitesh Shah <hitesh@apache.org> wrote:
> >
> > Another clarification - if the plan is to publish to the binaries after
> the
> > vote, I would recommend having the binaries be generated and made
> available
> > as part of the vote to verify that the contents/license/notice files are
> > correct.
> >
> > thanks
> > -- HItesh
> >
> >> On Thu, Aug 3, 2017 at 1:37 PM, Hitesh Shah <hitesh@apache.org> wrote:
> >>
> >> Thanks for the clarification, Ash.
> >>
> >> In that case, I am assuming that a release push should be okay as long
> as
> >> the package description clearly indicates "incubating" and the version
> is
> >> "1.8.2" (no rc suffix though - only a voted upon and verified release
> >> should be published). I would recommend calling this out in the IPMC
> vote
> >> to get general feedback if this has not been already addressed in other
> >> threads.
> >>
> >> thanks
> >> -- Hitesh
> >>
> >> On Thu, Aug 3, 2017 at 12:09 PM, Ash Berlin-Taylor <
> >> ash_airflowlist@firemirror.com> wrote:
> >>
> >>> pip can be very picky at times the version number of things it
> installs,
> >>> specifically when installing from wheels (which are pre-packaged, and
> >>> recommend because they are much faster to install.)
> >>>
> >>> A distribution version number should follow PEP-0440
> >>> https://www.python.org/dev/peps/pep-0440/ <
> https://www.python.org/dev/pe
> >>> ps/pep-0440/> which says:
> >>>
> >>>> # Public version identifiers
> >>>>
> >>>> The canonical public version identifiers MUST comply with the
> following
> >>> scheme:
> >>>>
> >>>>    [N!]N(.N)*[{a|b|rc}N][.postN][.devN]
> >>>
> >>> So according to PEP440 "incubating" MUST (in the RFC sence) not appear
> in
> >>> the version number.
> >>>
> >>>
> >>>
> >>>> On 3 Aug 2017, at 19:56, Hitesh Shah <hitesh@apache.org> wrote:
> >>>>
> >>>> -1
> >>>>
> >>>> Verified signatures and checksums.
> >>>> Verified Disclaimer.
> >>>> Ran rat-check
> >>>> Verified License for source. Quick question on the bundled images -
> are
> >>>> these covered by the License file?
> >>>> Version number incorrect and does not contain "incubating".
> >>>>
> >>>> The main (albeit a very minor change) issue is that the version in
> >>>> version.py is "1.8.2rc2". If there is a plan to publish the binary
> >>>> artifacts, the version should be "1.8.2-incubating". Changing this
> means
> >>>> that the tarball needs to be modified and would no longer be the
> >>> artifact
> >>>> that was voted upon.
> >>>>
> >>>> thanks
> >>>> -- Hitesh
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> On Wed, Aug 2, 2017 at 1:17 PM, Chris Riccomini <
> criccomini@apache.org>
> >>>> wrote:
> >>>>
> >>>>> Verified asc, sha, and md5. Thanks, Max!
> >>>>>
> >>>>> +1 (binding)
> >>>>>
> >>>>> On Wed, Aug 2, 2017 at 12:56 PM, Maxime Beauchemin <
> >>>>> maximebeauchemin@gmail.com> wrote:
> >>>>>
> >>>>>> It's the same, only the bundling is different. In this case we're
> >>>>> bundling
> >>>>>> the source with and INSTALL file as opposed to the SDIST binaries.
> >>>>>>
> >>>>>> If it goes well I don't mind doing 1.9.0 soon after.
> >>>>>>
> >>>>>> Max
> >>>>>>
> >>>>>> On Wed, Aug 2, 2017 at 12:37 PM, Bolke de Bruin <bdbruin@gmail.com>
> >>>>> wrote:
> >>>>>>
> >>>>>>> Assuming same as rc2 which we are running for over a month now.
> >>>>>>>
> >>>>>>> +1 (binding)
> >>>>>>>
> >>>>>>> Bolke
> >>>>>>>
> >>>>>>> Sent from my iPhone
> >>>>>>>
> >>>>>>>> On 1 Aug 2017, at 23:52, Maxime Beauchemin <
> >>>>> maximebeauchemin@gmail.com
> >>>>>>>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> 1.8.2 RC3 is baked and available at:
> >>>>>>>> https://dist.apache.org/repos/dist/dev/incubator/airflow, public
> >>>>> keys
> >>>>>>>> are available
> >>>>>>>> at https://dist.apache.org/repos/dist/release/incubator/airflow.
> >>>>>>>>
> >>>>>>>> This is a source release that comes with INSTALL instructions.
> >>>>>>>>
> >>>>>>>> 1.8.2 RC3 is build upon 1.8.1 with the commits listed bellow on
> top
> >>>>> of
> >>>>>>> it.
> >>>>>>>> I added the JIRAs that were identified blockers and targeted
> 1.8.2.
> >>> I
> >>>>>>>> attempted to bring in all of the JIRAs that targeted 1.8.2 but
> >>> bailed
> >>>>>> on
> >>>>>>>> the ones that were generating merge conflicts. I also added all of
> >>>>> the
> >>>>>>>> JIRAs that we've been running in production at Airbnb.
> >>>>>>>>
> >>>>>>>> Issues fixed:
> >>>>>>>> 9a53e66 [AIRFLOW-809][AIRFLOW-1] Use __eq__ ColumnOperator When
> >>>>> Testing
> >>>>>>>> Booleans
> >>>>>>>> 333e0b3 [AIRFLOW-1296] Propagate SKIPPED to all downstream tasks
> >>>>>>>> 93825d5 [AIRFLOW-XXX] Re-enable caching for hadoop components
> >>>>>>>> 33a9dcb [AIRFLOW-XXX] Pin Hive and Hadoop to a specific version
> and
> >>>>>>> create
> >>>>>>>> writable warehouse dir
> >>>>>>>> 7cff6cd [AIRFLOW-1308] Disable nanny usage for Dask
> >>>>>>>> 570b2ed [AIRFLOW-1294] Backfills can loose tasks to execute
> >>>>>>>> 3f48d48 [AIRFLOW-1291] Update NOTICE and LICENSE files to match
> ASF
> >>>>>>>> requirements
> >>>>>>>> 69bd269 [AIRFLOW-1160] Update Spark parameters for Mesos
> >>>>>>>> 9692510 [AIRFLOW 1149][AIRFLOW-1149] Allow for custom filters in
> >>>>> Jinja2
> >>>>>>>> templates
> >>>>>>>> 6de5330 [AIRFLOW-1119] Fix unload query so headers are on first
> >>> row[]
> >>>>>>>> b4e9eb8 [AIRFLOW-1089] Add Spark application arguments
> >>>>>>>> a4083f3 [AIRFLOW-1078] Fix latest_runs endpoint for old flask
> >>>>> versions
> >>>>>>>> 7a02841 [AIRFLOW-1074] Don't count queued tasks for concurrency
> >>>>> limits
> >>>>>>>> a2c18a5 [AIRFLOW-1064] Change default sort to job_id for
> >>>>>>>> TaskInstanceModelView
> >>>>>>>> d1c64ab [AIRFLOW-1038] Specify celery serialization options
> >>>>> explicitly
> >>>>>>>> b4ee88a [AIRFLOW-1036] Randomize exponential backoff
> >>>>>>>> 9fca409 [AIRFLOW-993] Update date inference logic
> >>>>>>>> 272c2f5 [AIRFLOW-1167] Support microseconds in FTPHook
> modification
> >>>>>> time
> >>>>>>>> c7c0b72 [AIRFLOW-1179] Fix Pandas 0.2x breaking Google BigQuery
> >>>>> change
> >>>>>>>> acd0166 [AIRFLOW-1263] Dynamic height for charts
> >>>>>>>> 7f33f6e [AIRFLOW-1266] Increase width of gantt y axis
> >>>>>>>> fc33c04 [AIRFLOW-1290] set docs author to 'Apache Airflow'
> >>>>>>>> 2e9eee3 [AIRFLOW-1282] Fix known event column sorting
> >>>>>>>> 2389a8a [AIRFLOW-1166] Speed up _change_state_for_tis_without_
> >>> dagrun
> >>>>>>>> bf966e6 [AIRFLOW-1192] Some enhancements to qubole_operator
> >>>>>>>> 57d5bcd [AIRFLOW-1281] Sort variables by key field by default
> >>>>>>>> 802fc15 [AIRFLOW-1244] Forbid creation of a pool with empty name
> >>>>>>>> 1232b6a [AIRFLOW-1243] DAGs table has no default entries to show
> >>>>>>>> b0ba3c9 [AIRFLOW-1227] Remove empty column on the Logs view
> >>>>>>>> c406652 [AIRFLOW-1226] Remove empty column on the Jobs view
> >>>>>>>> 51a83cc [AIRFLOW-1199] Fix create modal
> >>>>>>>> cac7d4c [AIRFLOW-1200] Forbid creation of a variable with an empty
> >>>>> key
> >>>>>>>> 5f3ee52 [AIRFLOW-1186] Sort dag.get_task_instances by
> execution_date
> >>>>>>>> f446c08 [AIRFLOW-1145] Fix closest_date_partition function with
> >>>>> before
> >>>>>>> set
> >>>>>>>> to True If we're looking for the closest date before, we should
> take
> >>>>>> the
> >>>>>>>> latest date in the list of date before.
> >>>>>>>> 93b8e96 [AIRFLOW-1180] Fix flask-wtf version for
> test_csrf_rejection
> >>>>>>>> bb56805 [AIRFLOW-1170] DbApiHook insert_rows inserts parameters
> >>>>>>> separately
> >>>>>>>> 093b2f0 [AIRFLOW-1150] Fix scripts execution in sparksql hook[]
> >>>>>>>> 777f181 [AIRFLOW-1168] Add closing() to all connections and
> cursors
> >>>>>>>>
> >>>>>>>> As part of the process I updated the release intructions here:
> >>>>>>>> https://cwiki.apache.org/confluence/display/AIRFLOW/
> >>>>> Releasing+Airflow
> >>>>>>>>
> >>>>>>>> Max
> >>>>>>>
> >>>>>>
> >>>>>
> >>>
> >>>
> >>
>

--f403045f4fa4272fb70555e1595e--