airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maxime Beauchemin <maximebeauche...@gmail.com>
Subject Re: [VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC4
Date Wed, 16 Aug 2017 22:19:24 GMT
Ok done, the whole process is very prone to error, hopefully I did
everything right.

Max

On Wed, Aug 16, 2017 at 2:41 PM, Ruslan Dautkhanov <dautkhanov@gmail.com>
wrote:

> +1
> On Wed, Aug 16, 2017 at 3:33 PM Bolke de Bruin <bdbruin@gmail.com> wrote:
>
> > Max
> >
> > Can you move this forward please? You have received 2 binding +1 if you
> > include your own it's 3 and the vote has been open long enough. Time to
> go
> > to the ipmc!
> >
> > Cheers
> > Bolke
> >
> > Sent from my iPhone
> >
> > > On 9 Aug 2017, at 20:06, Bolke de Bruin <bdbruin@gmail.com> wrote:
> > >
> > > We will maintain 1.8.X and 1.9.X until a next major release. So there
> > will be a 1.8.3 regardless of 1.9.
> > >
> > > Bolke
> > >
> > > Sent from my iPhone
> > >
> > >> On 9 Aug 2017, at 19:07, George Leslie-Waksman <
> george@cloverhealth.com.INVALID>
> > wrote:
> > >>
> > >> We've been having a lot of problems with 1.8.1 because of a bug in the
> > >> DagRun deadlock detection, reported under AIRFLOW-1420 and
> AIRFLOW-1473.
> > >>
> > >> I have a bugfix PR out:
> > >> https://github.com/apache/incubator-airflow/pull/2506
> > >>
> > >> I would love to see this bugfix make it in before work starts on new
> > >> features for 1.9.0. If it can make it in to 1.8.2, great; if not, I
> > would
> > >> hope that there can be a 1.8.3 with it sooner rather than later.
> > >>
> > >> --George
> > >>
> > >> On Wed, Aug 9, 2017 at 8:58 AM Maxime Beauchemin <
> > maximebeauchemin@gmail.com>
> > >> wrote:
> > >>
> > >>> I still consider 1.8.2 a bit of a practice run for me, once it's out
> > I'll
> > >>> kick off 1.9.0 fresh forked off master.
> > >>>
> > >>> Max
> > >>>
> > >>> On Wed, Aug 9, 2017 at 8:57 AM, Maxime Beauchemin <
> > >>> maximebeauchemin@gmail.com> wrote:
> > >>>
> > >>>> Oh I need to simplify the instructions and point to using that.
> > >>>>
> > >>>> Max
> > >>>>
> > >>>> On Wed, Aug 9, 2017 at 8:51 AM, Bolke de Bruin <bdbruin@gmail.com>
> > >>> wrote:
> > >>>>
> > >>>>> +1 (binding),
> > >>>>>
> > >>>>> You know that there is a sign.sh in the dev folder? Makes your
life
> > >>>>> easier :-).
> > >>>>>
> > >>>>>
> > >>>>> Sent from my iPhone
> > >>>>>
> > >>>>>> On 8 Aug 2017, at 19:37, Chris Riccomini <criccomini@apache.org>
> > >>> wrote:
> > >>>>>>
> > >>>>>> Gocha.
> > >>>>>>
> > >>>>>> +1 (binding)
> > >>>>>>
> > >>>>>> Validated artifacts:
> > >>>>>>
> > >>>>>> $ gpg --print-md SHA512 apache-airflow-1.8.2+
> incubating-bin.tar.gz
> > >>>>>> apache-airflow-1.8.2+incubating-source.tar.gz
> > >>>>>> apache-airflow-1.8.2+incubating-bin.tar.gz:
> > >>>>>> F32FFE95 BDC2066A 125624F1 62731E34 AD45D5A4 01B73345 FDD3FF09
> > >>> FFD9CEF8
> > >>>>>> 632611DF
> > >>>>>> B943FA79 9074B8AA 1B54616F EB47A3DB B35D740A 1DBB907D 32E83F59
> > >>>>>> apache-airflow-1.8.2+incubating-source.tar.gz:
> > >>>>>> 51381ED4 7FA0C0DE 5822BA23 B9EC7290 EA668259 6CD03800 B176E9A6
> > >>> 24F27955
> > >>>>>> 8CC0B1A8
> > >>>>>> 7DC89DF8 FF58A32F 20B5C448 FC7EECA5 17E0B749 6B356B42 05E020AB
> > >>>>>>
> > >>>>>> $ cat apache-airflow-1.8.2+incubating-bin.tar.gz.sha
> > >>>>>> apache-airflow-1.8.2+incubating-source.tar.gz.sha
> > >>>>>> apache-airflow-1.8.2+incubating-bin.tar.gz:
> > >>>>>> F32FFE95 BDC2066A 125624F1 62731E34 AD45D5A4 01B73345 FDD3FF09
> > >>> FFD9CEF8
> > >>>>>> 632611DF
> > >>>>>> B943FA79 9074B8AA 1B54616F EB47A3DB B35D740A 1DBB907D 32E83F59
> > >>>>>> apache-airflow-1.8.2+incubating-source.tar.gz:
> > >>>>>> 51381ED4 7FA0C0DE 5822BA23 B9EC7290 EA668259 6CD03800 B176E9A6
> > >>> 24F27955
> > >>>>>> 8CC0B1A8
> > >>>>>> 7DC89DF8 FF58A32F 20B5C448 FC7EECA5 17E0B749 6B356B42 05E020AB
> > >>>>>>
> > >>>>>> $ md5 apache-airflow-1.8.2+incubating-bin.tar.gz
> > >>>>>> apache-airflow-1.8.2+incubating-source.tar.gz
> > >>>>>> MD5 (apache-airflow-1.8.2+incubating-bin.tar.gz) =
> > >>>>>> 7e57eda714847f0057f3e31daf90a3d6
> > >>>>>> MD5 (apache-airflow-1.8.2+incubating-source.tar.gz) =
> > >>>>>> 62d371c2e828f6631e8d8646f09bf593
> > >>>>>>
> > >>>>>> $ cat apache-airflow-1.8.2+incubating-bin.tar.gz.md5
> > >>>>>> apache-airflow-1.8.2+incubating-source.tar.gz.md5
> > >>>>>> apache-airflow-1.8.2+incubating-bin.tar.gz:
> > >>>>>> 7E 57 ED A7 14 84 7F 00  57 F3 E3 1D AF 90 A3 D6
> > >>>>>> apache-airflow-1.8.2+incubating-source.tar.gz:
> > >>>>>> 62 D3 71 C2 E8 28 F6 63  1E 8D 86 46 F0 9B F5 93
> > >>>>>>
> > >>>>>> $ gpg --verify  apache-airflow-1.8.2+incubating-bin.tar.gz.asc
> > >>>>>> apache-airflow-1.8.2+incubating-bin.tar.gz
> > >>>>>> gpg: Signature made Mon Aug  7 22:03:07 2017 PDT using
RSA key ID
> > >>>>> C7BC7E0D
> > >>>>>> gpg: Good signature from "Maxime Beauchemin <
> > >>>>> maximebeauchemin@apache.org>"
> > >>>>>> [unknown]
> > >>>>>> gpg: WARNING: This key is not certified with a trusted
signature!
> > >>>>>> gpg:          There is no indication that the signature
belongs to
> > the
> > >>>>>> owner.
> > >>>>>> Primary key fingerprint: 99E2 0282 4A25 EAA3 7505  50BE
E6F0 505C
> > C7BC
> > >>>>> 7E0D
> > >>>>>>
> > >>>>>> $ gpg --verify apache-airflow-1.8.2rc3+incubating.tar.gz.asc
> > >>>>>> apache-airflow-1.8.2rc3+incubating.tar.gz
> > >>>>>> gpg: Signature made Tue Aug  1 12:43:42 2017 PDT using
RSA key ID
> > >>>>> C7BC7E0D
> > >>>>>> gpg: Good signature from "Maxime Beauchemin <
> > >>>>> maximebeauchemin@apache.org>"
> > >>>>>> [unknown]
> > >>>>>> gpg: WARNING: This key is not certified with a trusted
signature!
> > >>>>>> gpg:          There is no indication that the signature
belongs to
> > the
> > >>>>>> owner.
> > >>>>>> Primary key fingerprint: 99E2 0282 4A25 EAA3 7505  50BE
E6F0 505C
> > C7BC
> > >>>>> 7E0D
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> On Tue, Aug 8, 2017 at 10:35 AM, Maxime Beauchemin <
> > >>>>>> maximebeauchemin@gmail.com> wrote:
> > >>>>>>
> > >>>>>>> I was told to remove the `RCx` from the name and validated
> putting
> > >>>>> them in
> > >>>>>>> a subfolder. I'm guessing the idea is to be able to
just move the
> > >>>>> files to
> > >>>>>>> their final location without renaming them upon a successful
> vote.
> > >>>>>>>
> > >>>>>>> Max
> > >>>>>>>
> > >>>>>>> On Tue, Aug 8, 2017 at 10:26 AM, Chris Riccomini <
> > >>>>> criccomini@apache.org>
> > >>>>>>> wrote:
> > >>>>>>>
> > >>>>>>>> Are the artifacts supposed to include `rc4` in
their name? The
> rc3
> > >>> had
> > >>>>>>>> that, but rc4 appears to have file names with no
RC in the
> > filename.
> > >>>>>>>>
> > >>>>>>>> On Mon, Aug 7, 2017 at 10:23 PM, Maxime Beauchemin
<
> > >>>>>>>> maximebeauchemin@gmail.com> wrote:
> > >>>>>>>>
> > >>>>>>>>> 1.8.2 RC4 is baked and available at:
> > >>>>>>>>> https://dist.apache.org/repos/dist/dev/incubator/airflow,
> public
> > >>>>> keys
> > >>>>>>>>> are available at https://dist.apache.org/repos/
> > >>>>>>>> dist/release/incubator/airf
> > >>>>>>>>> low.
> > >>>>>>>>>
> > >>>>>>>>> apache-airflow-1.8.2+incubating-source.tar.gz
> > >>>>>>>>> <https://dist.apache.org/repos/dist/dev/incubator/
> > >>>>>>>> airflow/1.8.2rc4/apache-
> > >>>>>>>>> airflow-1.8.2+incubating-source.tar.gz>
> > >>>>>>>>> is
> > >>>>>>>>> a source release that comes with INSTALL instructions.
> > >>>>>>>>>
> > >>>>>>>>> Along with it, for convenience, find the binary
Python "sdist"
> as
> > >>>>>>>>> apache-airflow-1.8.2+incubating-bin.tar.gz
> > >>>>>>>>> <https://dist.apache.org/repos/dist/dev/incubator/
> > >>>>>>>> airflow/1.8.2rc4/apache-
> > >>>>>>>>> airflow-1.8.2+incubating-bin.tar.gz>
> > >>>>>>>>>
> > >>>>>>>>> 1.8.2 RC4 is build upon 1.8.1 with the commits
listed bellow on
> > top
> > >>>>> of
> > >>>>>>>> it.
> > >>>>>>>>> I added the JIRAs that were identified blockers
and targeted
> > >>> 1.8.2. I
> > >>>>>>>>> attempted to bring in all of the JIRAs that
targeted 1.8.2 but
> > >>> bailed
> > >>>>>>> on
> > >>>>>>>>> the ones that were generating merge conflicts.
I also added all
> > of
> > >>>>> the
> > >>>>>>>>> JIRAs that we've been running in production
at Airbnb.
> > >>>>>>>>>
> > >>>>>>>>> Issues fixed:
> > >>>>>>>>> 9a53e66 [AIRFLOW-809][AIRFLOW-1] Use __eq__
ColumnOperator When
> > >>>>> Testing
> > >>>>>>>>> Booleans
> > >>>>>>>>> 333e0b3 [AIRFLOW-1296] Propagate SKIPPED to
all downstream
> tasks
> > >>>>>>>>> 93825d5 [AIRFLOW-XXX] Re-enable caching for
hadoop components
> > >>>>>>>>> 33a9dcb [AIRFLOW-XXX] Pin Hive and Hadoop to
a specific version
> > and
> > >>>>>>>> create
> > >>>>>>>>> writable warehouse dir
> > >>>>>>>>> 7cff6cd [AIRFLOW-1308] Disable nanny usage
for Dask
> > >>>>>>>>> 570b2ed [AIRFLOW-1294] Backfills can loose
tasks to execute
> > >>>>>>>>> 3f48d48 [AIRFLOW-1291] Update NOTICE and LICENSE
files to match
> > ASF
> > >>>>>>>>> requirements
> > >>>>>>>>> 69bd269 [AIRFLOW-1160] Update Spark parameters
for Mesos
> > >>>>>>>>> 9692510 [AIRFLOW 1149][AIRFLOW-1149] Allow
for custom filters
> in
> > >>>>> Jinja2
> > >>>>>>>>> templates
> > >>>>>>>>> 6de5330 [AIRFLOW-1119] Fix unload query so
headers are on first
> > >>> row[]
> > >>>>>>>>> b4e9eb8 [AIRFLOW-1089] Add Spark application
arguments
> > >>>>>>>>> a4083f3 [AIRFLOW-1078] Fix latest_runs endpoint
for old flask
> > >>>>> versions
> > >>>>>>>>> 7a02841 [AIRFLOW-1074] Don't count queued tasks
for concurrency
> > >>>>> limits
> > >>>>>>>>> a2c18a5 [AIRFLOW-1064] Change default sort
to job_id for
> > >>>>>>>>> TaskInstanceModelView
> > >>>>>>>>> d1c64ab [AIRFLOW-1038] Specify celery serialization
options
> > >>>>> explicitly
> > >>>>>>>>> b4ee88a [AIRFLOW-1036] Randomize exponential
backoff
> > >>>>>>>>> 9fca409 [AIRFLOW-993] Update date inference
logic
> > >>>>>>>>> 272c2f5 [AIRFLOW-1167] Support microseconds
in FTPHook
> > modification
> > >>>>>>> time
> > >>>>>>>>> c7c0b72 [AIRFLOW-1179] Fix Pandas 0.2x breaking
Google BigQuery
> > >>>>> change
> > >>>>>>>>> acd0166 [AIRFLOW-1263] Dynamic height for charts
> > >>>>>>>>> 7f33f6e [AIRFLOW-1266] Increase width of gantt
y axis
> > >>>>>>>>> fc33c04 [AIRFLOW-1290] set docs author to 'Apache
Airflow'
> > >>>>>>>>> 2e9eee3 [AIRFLOW-1282] Fix known event column
sorting
> > >>>>>>>>> 2389a8a [AIRFLOW-1166] Speed up
> > >>> _change_state_for_tis_without_dagrun
> > >>>>>>>>> bf966e6 [AIRFLOW-1192] Some enhancements to
qubole_operator
> > >>>>>>>>> 57d5bcd [AIRFLOW-1281] Sort variables by key
field by default
> > >>>>>>>>> 802fc15 [AIRFLOW-1244] Forbid creation of a
pool with empty
> name
> > >>>>>>>>> 1232b6a [AIRFLOW-1243] DAGs table has no default
entries to
> show
> > >>>>>>>>> b0ba3c9 [AIRFLOW-1227] Remove empty column
on the Logs view
> > >>>>>>>>> c406652 [AIRFLOW-1226] Remove empty column
on the Jobs view
> > >>>>>>>>> 51a83cc [AIRFLOW-1199] Fix create modal
> > >>>>>>>>> cac7d4c [AIRFLOW-1200] Forbid creation of a
variable with an
> > empty
> > >>>>> key
> > >>>>>>>>> 5f3ee52 [AIRFLOW-1186] Sort dag.get_task_instances
by
> > >>> execution_date
> > >>>>>>>>> f446c08 [AIRFLOW-1145] Fix closest_date_partition
function with
> > >>>>> before
> > >>>>>>>> set
> > >>>>>>>>> to True If we're looking for the closest date
before, we should
> > >>> take
> > >>>>>>> the
> > >>>>>>>>> latest date in the list of date before.
> > >>>>>>>>> 93b8e96 [AIRFLOW-1180] Fix flask-wtf version
for
> > >>> test_csrf_rejection
> > >>>>>>>>> bb56805 [AIRFLOW-1170] DbApiHook insert_rows
inserts parameters
> > >>>>>>>> separately
> > >>>>>>>>> 093b2f0 [AIRFLOW-1150] Fix scripts execution
in sparksql hook[]
> > >>>>>>>>> 777f181 [AIRFLOW-1168] Add closing() to all
connections and
> > cursors
> > >>>>>>>>>
> > >>>>>>>>> As part of the process I updated the release
intructions here:
> > >>>>>>>>> https://cwiki.apache.org/confluence/display/AIRFLOW/Releasin
> > >>>>> g+Airflow
> > >>>>>>>>>
> > >>>>>>>>> Max
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>
> > >>>>
> > >>>>
> > >>>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message