airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maxime Beauchemin <>
Subject Re: Release Manager for 1.8.2?
Date Thu, 08 Jun 2017 15:42:10 GMT
I branched off `v1-8-test` so all should be good. I just didn't know if I
could move ahead with that branch just yet so I branched off. I just got
back on `v1-8-test` and pushed what I have to Apache [somehow I had to
rebase meaning someone added something over the past 24 hours].

I just set `Fix Version` of AIRFLOW-1294 to 1.8.2 and set it to blocker.

I cherry-picked all the commits targeting 1.8.2 I could without getting
into conflicts. I'd only work at resolving conflicts on blockers. My plan
is to add only the 2 rows in red for RC1.

Here's the current output of `airflow-jira 1.8.2`:

                                   |MERGED|PR    |COMMITAIRFLOW-1294
   |Bug         ||Blocker   ||Open      |Backfills can loose tasks to
execute due to tasks |0     |-     |-
AIRFLOW-1291      |Bug         ||Blocker     ||Open      |Update
NOTICE and LICENSE files to meet ASF specs |0     |-     |-
AIRFLOW-1290      |Bug         ||Major     ||Resolved  |Change docs
author from "Maxime Beauchemin" to "Ap|1     |#na
AIRFLOW-1282      |Bug         ||Major     ||Resolved  |Fix known
event column sorting                    |1     |#2350
AIRFLOW-1281      |Bug         ||Minor     ||Resolved  |Sort variables
by key field by default            |1     |#2347
AIRFLOW-1275      |Bug         ||Major     ||In Progress|Fix `airflow
pool` command exception              |0     |-     |-
AIRFLOW-1274      |Bug         ||Major     ||Open      |HttpSensor
parameter params is overriding BaseOper|0     |-     |-
AIRFLOW-1266      |Bug         ||Major     ||Open      |Long task
names are truncated in gannt view       |1     |#2345
AIRFLOW-1263      |Bug         ||Major     ||Resolved  |Airflow Charts
Pages Should Have Dynamic Heights  |1     |#2344
AIRFLOW-1244      |Bug         ||Major     ||Resolved  |Forbid
creation of a pool with empty name         |1     |#2324
AIRFLOW-1243      |Bug         ||Minor     ||Resolved  |DAGs table has
no default entries to show         |1     |#2323
AIRFLOW-1227      |Bug         ||Minor     ||Resolved  |Remove empty
column on the Logs view              |1     |#2310
AIRFLOW-1226      |Bug         ||Minor     ||Resolved  |Remove empty
column on the Jobs view              |1     |#2309
AIRFLOW-1221      |Bug         ||Major     ||Resolved  |Fix
DatabricksSubmitRunOperator Templating        |0     |-     |-
AIRFLOW-1217      |Bug         ||Major     ||Resolved  |Enable logging
in Sqoop hook                      |0     |-     |-
AIRFLOW-1213      |Bug         ||Major     ||Resolved  |Add hcatalog
parameters to the sqoop operator/hook|0     |-     |-
AIRFLOW-1203      |Bug         ||Major     ||Closed    |Tests failing
after oauth upgrade                 |0     |-     |-
AIRFLOW-1201      |Bug         ||Major     ||Resolved  |Update
deprecated 'nose-parameterized' library to |0     |-     |-
AIRFLOW-1200      |Bug         ||Major     ||Resolved  |Forbid
creation of a variable with an empty key   |1     |#2299
AIRFLOW-1199      |Bug         ||Major     ||Resolved  |Create modal
looks like edit modal (edit tab selec|1     |#2293
AIRFLOW-1197      |Bug         ||Major     ||Open      |Contrib Spark
Submit Hook on_kill error           |0     |-     |-
AIRFLOW-1192      |New Feature ||Major     ||Resolved  |Add support to
run Zeppelin Notebooks with Qubole |1     |#2322
AIRFLOW-1191      |Improvement ||Major     ||Open      |Contrib Spark
Submit hook should permit override o|0     |-     |-
AIRFLOW-1186      |Bug         ||Major     ||Resolved  |Task duration
shows weird graph for some dags     |1     |#2284
AIRFLOW-1180      |Bug         ||Major     ||Resolved  |Fix flask-wtf
version in for test_csrf_re|1     |#2280
AIRFLOW-1179      |Bug         ||Major     ||Closed    |Pandas 0.20
broke Google BigQuery hook            |1     |#2279
AIRFLOW-1170      |Bug         ||Major     ||Resolved  |DbApiHook
insert rows does not use prepared statem|1     |#2270
AIRFLOW-1168      |Bug         ||Major     ||Resolved  |DbApiHook does
not close connections when an excep|1     |#2269
AIRFLOW-1167      |Bug         ||Major     ||Resolved  |FTPHook file
modification date does not support op|1     |#2268
AIRFLOW-1166      |Bug         ||Major     ||Resolved
|_change_state_for_tis_without_dagrun can take very|1     |#2267
AIRFLOW-1160      |Bug         ||Major     ||Resolved  |Upadte
SparkSubmitOperator parameters             |1     |#2265
AIRFLOW-1150      |Bug         ||Major     ||Resolved  |Fix scripts
execution in SparkSql hook            |1     |#2259
AIRFLOW-1149      |Bug         ||Major     ||Resolved  |Allow custom
filters to be added to jinja2        |1     |#2258
AIRFLOW-1145      |Bug         ||Minor     ||Resolved
|Closest_date_partition not working with before = T|1     |#2257
AIRFLOW-1136      |Bug         ||Minor     ||Resolved  |Invalid
parameters are not captured for Sqoop oper|0     |-     |-
AIRFLOW-1119      |Bug         ||Major     ||Resolved  |Redshift to S3
operator - headers not on first row|1     |#2245
AIRFLOW-1089      |Improvement ||Major     ||Resolved  |Add Spark
application arguments to SparkSubmitOper|1     |#2229
AIRFLOW-1081      |Bug         ||Major     ||Resolved  |Task duration
page is slow                        |0     |-     |-
AIRFLOW-1078      |Bug         ||Major     ||Resolved  |Latest_runs
endpoint broken in old flask versions |1     |#2224
AIRFLOW-1074      |Bug         ||Major     ||Open      |Do not count
queued tasks in scheduler concurrency|1     |#2221
AIRFLOW-1064      |Improvement ||Major     ||Resolved
|TaskInstanceModelView is slow                     |1     |#2215
AIRFLOW-1047      |Bug         ||Major     ||Resolved  |Airflow logs
vulnerable to XSS                    |0     |-     |-
AIRFLOW-1041      |Bug         ||Major     ||Resolved  |DockerOperator
replaces its xcom_push method with |0     |-     |-
AIRFLOW-1038      |Bug         ||Major     ||Resolved  |Specify celery
serializers explicitly and pin vers|1     |#na
AIRFLOW-1036      |Improvement ||Major     ||Resolved  |Exponential
backoff should use randomization      |1     |#2262
AIRFLOW-1007      |Bug         ||Major     ||Resolved  |Jinja sandbox
is vulnerable to RCE                |0     |-     |-
AIRFLOW-993       |Bug         ||Minor     ||Resolved  |Dags should
modify the start date and end date of |1     |#2157
AIRFLOW-945       |Bug         ||Major     ||Resolved  |Revert
psycopg2 workaround when psycopg2 2.7.1 is |0     |-     |-
AIRFLOW-935       |Bug         ||Major     ||Resolved  |Impossible to
use plugin executors                |0     |-     |-
AIRFLOW-860       |Bug         ||Major     ||Resolved  |Circular
module dependency prevents loading of cus|0     |-     |-


On Thu, Jun 8, 2017 at 12:18 AM, Bolke de Bruin <> wrote:

> Hi Max,
> Sounds good. Couple of things:
> * Can I suggest using the v1-8-test branch as the branch to be used for
> preparing the rc? If we hit RC then move it over to v1-8-stable? V1-8-test
> already had some fixes in that should land in 1.8.2 and the RC should be
> tagged in the stable branch. That also reduces to amount of merge conflicts
> probably as many have been merged. Where did you branch off from? Anyways,
> see also the release management thing on the wiki.
> Blocker(!)
> * In the backfills we can loose tasks to execute due to a task setting its
> own state to NONE if concurrency limits are reached, this makes them fall
> outside of the scope the backfill is managing hence they will not be
> executed ( <
>>). Setting itself to
> NONE should probably be “CONCURRENCY_REACHED” (new state). I have marked it
> as a blocker as we had multiple people hitting the issue, but I need 1-2
> days to get a patch. Feel free to downgrade to critical if you like :).
> Cheers
> Bolke
> > On 8 Jun 2017, at 02:35, Maxime Beauchemin <>
> wrote:
> >
> > What a pleasant, mind numbing afternoon doing some release management
> >
> > Notes:
> > * Added a warning that the package name has changed on Pypi
> > <>
> > * Removed references to my name here
> > <> and merged
> > * Addressed John D. Ament's concerns here
> > <>, please review!
> > * "footable" appears to have been removed, not a problem anymore
> > * that `airflow-jira is a god send! thanks Bolke.
> > * reviewed list of Airbnb's production cherries and flagged those as `Fix
> > Version == 1.8.2`
> > * Started branch v1-8-2.rc1 and started picking cherries using
> > `airflow-jira compare 1.8.2`
> >
> > I'll finish going through picking everything that targeted 1.8.2 that
> does
> > not create merge conflict.
> >
> > If there's anything flagged as "blocker" that generates merge conflict,
> > I'll go case by case about it.
> >
> > Soon after, I should be able to announce 1.8.2 RC1, hopefully sometime
> > tomorrow or Friday.
> >
> > Let me know if there's anything else I'm missing that I should consider.
> >
> > Cheers!
> >
> > Max

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message