airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maxime Beauchemin <maximebeauche...@gmail.com>
Subject Re: Release Manager for 1.8.2?
Date Fri, 23 Jun 2017 21:10:01 GMT
It feels so retro to type svn commands. Muscle memory wants me to write
`git`  so bad.

Ok so I removed all but apache-airflow-1.8.2rc2+incubating.tar.gz.* here:
https://dist.apache.org/repos/dist/dev/incubator/airflow/

Max

On Fri, Jun 23, 2017 at 1:17 PM, Chris Riccomini <criccomini@apache.org>
wrote:

> MD5 looks good:
>
> chrisr-00532:airflow chrisr$ md5 apache-airflow-1.8.2rc2+incubating.tar.gz
> MD5 (apache-airflow-1.8.2rc2+incubating.tar.gz) =
> 08937631caa976c507bdf48db22f1ca3
> chrisr-00532:airflow chrisr$ cat
> apache-airflow-1.8.2rc2+incubating.tar.gz.md5
> apache-airflow-1.8.2rc2+incubating.tar.gz:
> 08 93 76 31 CA A9 76 C5  07 BD F4 8D B2 2F 1C A3
>
> SHA looks good:
>
> chrisr-00532:airflow chrisr$ gpg --print-md SHA512
> apache-airflow-1.8.2rc2+incubating.tar.gz
> apache-airflow-1.8.2rc2+incubating.tar.gz:
> DA650C6A 1E42CB59 AE233239 1AC16C01 CD73262A 93F3082D 979C7177 58011B71
> DE66C802
>  90E3FBB9 B3C965AF 97DF32A7 CEBF6900 0600921B 8636FE3D 64E0CFDA
> chrisr-00532:airflow chrisr$ cat
> apache-airflow-1.8.2rc2+incubating.tar.gz.sha
> apache-airflow-1.8.2rc2+incubating.tar.gz:
> DA650C6A 1E42CB59 AE233239 1AC16C01 CD73262A 93F3082D 979C7177 58011B71
> DE66C802
>  90E3FBB9 B3C965AF 97DF32A7 CEBF6900 0600921B 8636FE3D 64E0CFDA
>
> ASC passed too.
>
> :thumbsup:
>
>
> On Fri, Jun 23, 2017 at 11:33 AM, Bolke de Bruin <bdbruin@gmail.com>
> wrote:
>
> > Yes.
> >
> > Bolke.
> >
> > > On 23 Jun 2017, at 19:55, Maxime Beauchemin <
> maximebeauchemin@gmail.com>
> > wrote:
> > >
> > > I appended my public key to the KEYS file.
> > >
> > > I'm not sure if I understand your 3rd point, should I remove all the
> > > previous tarballs and related files from
> https://dist.apache.org/repos/
> > > dist/dev/incubator/airflow/ ?
> > >
> > > Max
> > >
> > > On Fri, Jun 23, 2017 at 5:48 AM, Bolke de Bruin <bdbruin@gmail.com>
> > wrote:
> > >
> > >> Hi Max,
> > >>
> > >> Looking on a phone, so I might not have spotted everything.
> > >>
> > >> - Tag looks good
> > >> - Your public GPG key has not been added yet to
> > >> https://dist.apache.org/repos/dist/release/incubator/airflow/KEYS <
> > >> https://dist.apache.org/repos/dist/release/incubator/airflow/KEYS>
> > >> - Please remove the old tar balls from https://dist.apache.org/repos/
> > >> dist/dev/incubator/airflow/ <https://dist.apache.org/
> > >> repos/dist/dev/incubator/airflow/> is causes confusion for some
> reason
> > at
> > >> the IPMC
> > >>
> > >> @chris please take a look as well.
> > >>
> > >> Bolke
> > >>
> > >>> On 23 Jun 2017, at 00:34, Maxime Beauchemin <
> > maximebeauchemin@gmail.com>
> > >> wrote:
> > >>>
> > >>> Allllllllright, so I cherry-picked that last one, some of the other
> > last
> > >>> minute ones had conflicts, and I bailed on all conflictual
> non-blocking
> > >>> issues.
> > >>>
> > >>> So here is the new tag, it's on the v1-8-test branch:
> > >>> https://github.com/apache/incubator-airflow/releases/tag/1.8.2rc2
> > >>>
> > >>> The tarball, hashes and signature are here:
> > >>> https://dist.apache.org/repos/dist/dev/incubator/airflow/
> > >>>
> > >>> Changelog:
> > >>> https://github.com/apache/incubator-airflow/blob/1.8.
> > 2rc2/CHANGELOG.txt
> > >>>
> > >>> Can someone confirm this looks alright before I send an announcement
> > >> about
> > >>> 1.8.2rc2 to the list?
> > >>>
> > >>> Max
> > >>>
> > >>> On Wed, Jun 21, 2017 at 10:44 PM, Bolke de Bruin <bdbruin@gmail.com>
> > >> wrote:
> > >>>
> > >>>> Please also include https://github.com/apache/
> > >> incubator-airflow/pull/2022
> > >>>>
> > >>>> I have had reports it was also required for MySQL.
> > >>>>
> > >>>> Bolke
> > >>>>
> > >>>> Sent from my iPhone
> > >>>>
> > >>>>> On 22 Jun 2017, at 00:03, Maxime Beauchemin <
> > >> maximebeauchemin@gmail.com>
> > >>>> wrote:
> > >>>>>
> > >>>>> Ok I'll try to cherry-pick all of these, though if there are any
> > merge
> > >>>>> conflicts I'll pass and move their Fix-Version to 1.9.0.
> > >>>>>
> > >>>>> Max
> > >>>>>
> > >>>>>> On Wed, Jun 21, 2017 at 1:20 AM, Bolke de Bruin <
> bdbruin@gmail.com>
> > >>>> wrote:
> > >>>>>>
> > >>>>>> Hi max,
> > >>>>>>
> > >>>>>> PR 2365 is merged into master and also into v1-8-test.
> > >>>>>>
> > >>>>>> I have added:
> > >>>>>>
> > >>>>>> https://issues.apache.org/jira/browse/AIRFLOW-935 <
> > >>>>>> https://issues.apache.org/jira/browse/AIRFLOW-935> and
> > >>>>>> https://issues.apache.org/jira/browse/AIRFLOW-860 <
> > >>>>>> https://issues.apache.org/jira/browse/AIRFLOW-860>
> > >>>>>>
> > >>>>>> To the list fro 1.8.2. The PR for this has been in master for some
> > >> time:
> > >>>>>> https://github.com/apache/incubator-airflow/pull/2120 <
> > >>>>>> https://github.com/apache/incubator-airflow/pull/2120>
> > >>>>>>
> > >>>>>> Can you consider it for 1.8.2? Basically I promised to push to the
> > >>>> matter
> > >>>>>> :-).
> > >>>>>>
> > >>>>>> No blockers anymore as far as I know for 1.8.2.
> > >>>>>>
> > >>>>>> Bolke
> > >>>>>>
> > >>>>>> P.S. A friendly request if any of the committers do +1 or a LGTM
> to
> > a
> > >>>> PR,
> > >>>>>> please also merge the PR instead of leaving it to others.
> > “airflow-pr
> > >>>> merge
> > >>>>>> XXX” is there to help.
> > >>>>>>
> > >>>>>>> On 16 Jun 2017, at 15:11, Bolke de Bruin <bdbruin@gmail.com>
> > wrote:
> > >>>>>>>
> > >>>>>>> It is now pinned. I think Cloudera messed up their release for
> some
> > >>>>>> reason.
> > >>>>>>>
> > >>>>>>> Builds are succeeding again.
> > >>>>>>>
> > >>>>>>> Please note I have one blocker out for 1.8.2:
> > >>>> https://github.com/apache/
> > >>>>>> incubator-airflow/pull/2365 <https://github.com/apache/
> > >>>>>> incubator-airflow/pull/2365>
> > >>>>>>>
> > >>>>>>> Bolke
> > >>>>>>>
> > >>>>>>>> On 15 Jun 2017, at 16:43, Maxime Beauchemin <
> > >>>> maximebeauchemin@gmail.com
> > >>>>>> <mailto:maximebeauchemin@gmail.com>> wrote:
> > >>>>>>>>
> > >>>>>>>> Awesome, thanks for looking into it. Is there a way we can pin
> the
> > >>>>>> Cloudera
> > >>>>>>>> tarball to avoid surprises like this?
> > >>>>>>>>
> > >>>>>>>> On Thu, Jun 15, 2017 at 7:17 AM, Bolke de Bruin <
> > bdbruin@gmail.com
> > >>>>>> <mailto:bdbruin@gmail.com>> wrote:
> > >>>>>>>>
> > >>>>>>>>> Found the issue for the permission denied part: beeline is not
> > >>>>>> executable
> > >>>>>>>>> in the latest tarball from Cloudera. I’ll have a workaround for
> > >> this
> > >>>>>> in a
> > >>>>>>>>> few minutes and will push it to Apache right away.
> > >>>>>>>>>
> > >>>>>>>>> Bolke.
> > >>>>>>>>>
> > >>>>>>>>>> On 14 Jun 2017, at 18:15, Bolke de Bruin <bdbruin@gmail.com
> > >>>> <mailto:
> > >>>>>> bdbruin@gmail.com>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>> Hi Max,
> > >>>>>>>>>>
> > >>>>>>>>>> I have created https://github.com/apache/
> > >>>> incubator-airflow/pull/2365
> > >>>>>> <https://github.com/apache/incubator-airflow/pull/2365> <
> > >>>>>>>>> https://github.com/apache/incubator-airflow/pull/2365 <
> > >>>>>> https://github.com/apache/incubator-airflow/pull/2365>> for
> > >>>> AIRFLOW-1296,
> > >>>>>>>>> which I think should be treated as a blocker.
> > >>>>>>>>>>
> > >>>>>>>>>> I think Travis’ is failing due to a dependency upgrade of
> Dask.
> > >>>>>>>>>>
> > >>>>>>>>>> The permission denied error seems to come from a new Travis
> > config
> > >>>>>> that
> > >>>>>>>>> does not allow the creation of “/user/hive/warehouse” by the
> > normal
> > >>>>>> user.
> > >>>>>>>>> Probably a “sudo mkdir “ and a “sudo chown” will help here. Let
> > me
> > >>>>>> check.
> > >>>>>>>>>>
> > >>>>>>>>>> Bolke
> > >>>>>>>>>>
> > >>>>>>>>>>> On 14 Jun 2017, at 00:26, Maxime Beauchemin <
> > >>>>>> maximebeauchemin@gmail.com <mailto:maximebeauchemin@gmail.com>
> > >>>>>>>>> <mailto:maximebeauchemin@gmail.com <mailto:maximebeauchemin@
> > >>>> gmail.com>>>
> > >>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>> A quick update on my progress.
> > >>>>>>>>>>>
> > >>>>>>>>>>> I ran through the "Releasing Airflow" wiki playbook only to
> > >>>> realize I
> > >>>>>>>>> had
> > >>>>>>>>>>> published a version that failed the Travis build
> > >>>>>>>>>>> <https://travis-ci.org/mistercrunch/incubator-
> > >>>> airflow/jobs/242532965
> > >>>>>> <https://travis-ci.org/mistercrunch/incubator-
> > airflow/jobs/242532965>
> > >> <
> > >>>>>>>>> https://travis-ci.org/mistercrunch/incubator-
> > >> airflow/jobs/242532965
> > >>>> <
> > >>>>>> https://travis-ci.org/mistercrunch/incubator-
> airflow/jobs/242532965
> > >>>>>
> > >>>>>>>>>>> afterwards. I've
> > >>>>>>>>>>> been updating the wiki as I go and it seems like it will be
> > >> better
> > >>>>>> the
> > >>>>>>>>> next
> > >>>>>>>>>>> time around. I'm getting "Permission denied" on the Hive hook
> > >>>>>>>>> subprocess if
> > >>>>>>>>>>> anyone can shed some light on that.
> > >>>>>>>>>>>
> > >>>>>>>>>>> So I need to fix the unit tests at this point, and probably
> go
> > >>>>>> straight
> > >>>>>>>>> to
> > >>>>>>>>>>> `rc2` to avoid changing the files out there for `rc1`.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Max
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Thu, Jun 8, 2017 at 4:54 PM, Ruslan Dautkhanov <
> > >>>>>> dautkhanov@gmail.com <mailto:dautkhanov@gmail.com>
> > >>>>>>>>> <mailto:dautkhanov@gmail.com <mailto:dautkhanov@gmail.com>>>
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> It would be great if somebody would have a look at
> following 3
> > >>>> jiras
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> I've flagged
> > >>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013 <
> > >>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013> <
> > >>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013 <
> > >>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013>>
> > >>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1178 <
> > >>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1178>
> > >>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1055
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Two of them were Blockers for one of previous 1.8 releases
> but
> > >>>> were
> > >>>>>>>>>>>> de-escalated because no resources to fix them.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> I'm currently Assignee on AIRFLOW-1055 - can't unassign
> > myself -
> > >>>>>> feel
> > >>>>>>>>> free
> > >>>>>>>>>>>> to scratch my name there.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> --
> > >>>>>>>>>>>> Ruslan Dautkhanov
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Fri, Jun 9, 2017 at 12:23 AM, Maxime Beauchemin <
> > >>>>>>>>>>>> maximebeauchemin@gmail.com <mailto:maximebeauchemin@
> gmail.com
> > >>
> > >>>>>> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> Cool. I'll let everyone know when RC1 is out.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> @community, if you think anything else should be a blocker
> > for
> > >>>>>> 1.8.2
> > >>>>>>>>>>>> please
> > >>>>>>>>>>>>> flag it in Jira very soon or it will miss the release
> train!
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Max
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On Thu, Jun 8, 2017 at 9:06 AM, Chris Riccomini <
> > >>>>>>>>> criccomini@apache.org <mailto:criccomini@apache.org>>
> > >>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Thanks for doing all this, Max! When you're ready, I can
> > >> deploy
> > >>>> to
> > >>>>>>>>> some
> > >>>>>>>>>>>>> of
> > >>>>>>>>>>>>>> our environments to verify. Just let me know.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On Thu, Jun 8, 2017 at 8:42 AM, Maxime Beauchemin <
> > >>>>>>>>>>>>>> maximebeauchemin@gmail.com <mailto:maximebeauchemin@
> > gmail.com
> > >>>>
> > >>>>>> wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> I branched off `v1-8-test` so all should be good. I just
> > >> didn't
> > >>>>>> know
> > >>>>>>>>>>>>> if I
> > >>>>>>>>>>>>>>> could move ahead with that branch just yet so I branched
> > >> off. I
> > >>>>>> just
> > >>>>>>>>>>>>> got
> > >>>>>>>>>>>>>>> back on `v1-8-test` and pushed what I have to Apache
> > >> [somehow I
> > >>>>>> had
> > >>>>>>>>>>>> to
> > >>>>>>>>>>>>>>> rebase meaning someone added something over the past 24
> > >> hours].
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> I just set `Fix Version` of AIRFLOW-1294 to 1.8.2 and set
> > it
> > >> to
> > >>>>>>>>>>>>> blocker.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> I cherry-picked all the commits targeting 1.8.2 I could
> > >> without
> > >>>>>>>>>>>> getting
> > >>>>>>>>>>>>>>> into conflicts. I'd only work at resolving conflicts on
> > >>>>>> blockers. My
> > >>>>>>>>>>>>> plan
> > >>>>>>>>>>>>>>> is to add only the 2 rows in red for RC1.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Here's the current output of `airflow-jira 1.8.2`:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> ISSUE ID          |TYPE        ||PRIORITY  ||STATUS
> > >>>>>> |DESCRIPTION
> > >>>>>>>>>>>>>>>                               |MERGED|PR
> > >>>>>> |COMMITAIRFLOW-1294
> > >>>>>>>>>>>>>>> |Bug         ||Blocker   ||Open      |Backfills can loose
> > >> tasks
> > >>>>>> to
> > >>>>>>>>>>>>>>> execute due to tasks |0     |-     |-
> > >>>>>>>>>>>>>>> AIRFLOW-1291      |Bug         ||Blocker     ||Open
> > >>>> |Update
> > >>>>>>>>>>>>>>> NOTICE and LICENSE files to meet ASF specs |0     |-
>  |-
> > >>>>>>>>>>>>>>> AIRFLOW-1290      |Bug         ||Major     ||Resolved
> > >> |Change
> > >>>>>> docs
> > >>>>>>>>>>>>>>> author from "Maxime Beauchemin" to "Ap|1     |#na
> > >>>>>>>>>>>>>>> |fc33c040605e7b0121739af841790336bdfa3948
> > >>>>>>>>>>>>>>> AIRFLOW-1282      |Bug         ||Major     ||Resolved
> |Fix
> > >>>> known
> > >>>>>>>>>>>>>>> event column sorting                    |1     |#2350
> > >>>>>>>>>>>>>>> |2e9eee3697f6261de61c77ba584ace14b22751f0
> > >>>>>>>>>>>>>>> AIRFLOW-1281      |Bug         ||Minor     ||Resolved
> > |Sort
> > >>>>>>>>>>>> variables
> > >>>>>>>>>>>>>>> by key field by default            |1     |#2347
> > >>>>>>>>>>>>>>> |57d5bcdad4202fd8a94be07b4aeacee065801481
> > >>>>>>>>>>>>>>> AIRFLOW-1275      |Bug         ||Major     ||In
> > Progress|Fix
> > >>>>>>>>> `airflow
> > >>>>>>>>>>>>>>> pool` command exception              |0     |-     |-
> > >>>>>>>>>>>>>>> AIRFLOW-1274      |Bug         ||Major     ||Open
> > >>>>>> |HttpSensor
> > >>>>>>>>>>>>>>> parameter params is overriding BaseOper|0     |-     |-
> > >>>>>>>>>>>>>>> AIRFLOW-1266      |Bug         ||Major     ||Open
> > |Long
> > >>>> task
> > >>>>>>>>>>>>>>> names are truncated in gannt view       |1     |#2345
> > >>>>>>>>>>>>>>> |7f33f6e2fe4149e547860fdb237da26a5b7023ec
> > >>>>>>>>>>>>>>> AIRFLOW-1263      |Bug         ||Major     ||Resolved
> > >> |Airflow
> > >>>>>>>>>>>> Charts
> > >>>>>>>>>>>>>>> Pages Should Have Dynamic Heights  |1     |#2344
> > >>>>>>>>>>>>>>> |acd0166d8c7f5639898791848d985f6441b3b5b5
> > >>>>>>>>>>>>>>> AIRFLOW-1244      |Bug         ||Major     ||Resolved
> > >> |Forbid
> > >>>>>>>>>>>>>>> creation of a pool with empty name         |1     |#2324
> > >>>>>>>>>>>>>>> |802fc1549d0f173c646794f2476156d34383adc0
> > >>>>>>>>>>>>>>> AIRFLOW-1243      |Bug         ||Minor     ||Resolved
> > |DAGs
> > >>>>>> table
> > >>>>>>>>>>>> has
> > >>>>>>>>>>>>>>> no default entries to show         |1     |#2323
> > >>>>>>>>>>>>>>> |1232b6a4fd691e0faf081d4cf792a6d96d33b595
> > >>>>>>>>>>>>>>> AIRFLOW-1227      |Bug         ||Minor     ||Resolved
> > >> |Remove
> > >>>>>> empty
> > >>>>>>>>>>>>>>> column on the Logs view              |1     |#2310
> > >>>>>>>>>>>>>>> |b0ba3c91044b2abd92289d9b3f7d913060ef6c31
> > >>>>>>>>>>>>>>> AIRFLOW-1226      |Bug         ||Minor     ||Resolved
> > >> |Remove
> > >>>>>> empty
> > >>>>>>>>>>>>>>> column on the Jobs view              |1     |#2309
> > >>>>>>>>>>>>>>> |c406652bd4c8f8099841c69f2d1ccc9678d77bfd
> > >>>>>>>>>>>>>>> AIRFLOW-1221      |Bug         ||Major     ||Resolved
> |Fix
> > >>>>>>>>>>>>>>> DatabricksSubmitRunOperator Templating        |0     |-
> > >> |-
> > >>>>>>>>>>>>>>> AIRFLOW-1217      |Bug         ||Major     ||Resolved
> > >> |Enable
> > >>>>>>>>>>>> logging
> > >>>>>>>>>>>>>>> in Sqoop hook                      |0     |-     |-
> > >>>>>>>>>>>>>>> AIRFLOW-1213      |Bug         ||Major     ||Resolved
> |Add
> > >>>>>> hcatalog
> > >>>>>>>>>>>>>>> parameters to the sqoop operator/hook|0     |-     |-
> > >>>>>>>>>>>>>>> AIRFLOW-1203      |Bug         ||Major     ||Closed
> > |Tests
> > >>>>>>>>> failing
> > >>>>>>>>>>>>>>> after oauth upgrade                 |0     |-     |-
> > >>>>>>>>>>>>>>> AIRFLOW-1201      |Bug         ||Major     ||Resolved
> > >> |Update
> > >>>>>>>>>>>>>>> deprecated 'nose-parameterized' library to |0     |-
>  |-
> > >>>>>>>>>>>>>>> AIRFLOW-1200      |Bug         ||Major     ||Resolved
> > >> |Forbid
> > >>>>>>>>>>>>>>> creation of a variable with an empty key   |1     |#2299
> > >>>>>>>>>>>>>>> |cac7d4cb571a4c1f7ba53360765ea6c640c719df
> > >>>>>>>>>>>>>>> AIRFLOW-1199      |Bug         ||Major     ||Resolved
> > >> |Create
> > >>>>>> modal
> > >>>>>>>>>>>>>>> looks like edit modal (edit tab selec|1     |#2293
> > >>>>>>>>>>>>>>> |51a83cce96d41eb1b9df9f82188c36c7e12061df
> > >>>>>>>>>>>>>>> AIRFLOW-1197      |Bug         ||Major     ||Open
> > >> |Contrib
> > >>>>>>>>> Spark
> > >>>>>>>>>>>>>>> Submit Hook on_kill error           |0     |-     |-
> > >>>>>>>>>>>>>>> AIRFLOW-1192      |New Feature ||Major     ||Resolved
> |Add
> > >>>>>> support
> > >>>>>>>>>>>> to
> > >>>>>>>>>>>>>>> run Zeppelin Notebooks with Qubole |1     |#2322
> > >>>>>>>>>>>>>>> |bf966e63be2deb19fa2824efdfe854736318f1b5
> > >>>>>>>>>>>>>>> AIRFLOW-1191      |Improvement ||Major     ||Open
> > >> |Contrib
> > >>>>>>>>> Spark
> > >>>>>>>>>>>>>>> Submit hook should permit override o|0     |-     |-
> > >>>>>>>>>>>>>>> AIRFLOW-1186      |Bug         ||Major     ||Resolved
> > |Task
> > >>>>>>>>> duration
> > >>>>>>>>>>>>>>> shows weird graph for some dags     |1     |#2284
> > >>>>>>>>>>>>>>> |5f3ee52da7ca4f1fca8a84b3281b31aa6241514e
> > >>>>>>>>>>>>>>> AIRFLOW-1180      |Bug         ||Major     ||Resolved
> |Fix
> > >>>>>>>>> flask-wtf
> > >>>>>>>>>>>>>>> version in setup.py for test_csrf_re|1     |#2280
> > >>>>>>>>>>>>>>> |93b8e9643cc273ec0ecc063b9f851d9a8012cd03
> > >>>>>>>>>>>>>>> AIRFLOW-1179      |Bug         ||Major     ||Closed
> > >> |Pandas
> > >>>>>> 0.20
> > >>>>>>>>>>>>>>> broke Google BigQuery hook            |1     |#2279
> > >>>>>>>>>>>>>>> |c7c0b72d934fec10d890f7a3bd8fd06c0e2f44bd
> > >>>>>>>>>>>>>>> AIRFLOW-1170      |Bug         ||Major     ||Resolved
> > >>>> |DbApiHook
> > >>>>>>>>>>>>>>> insert rows does not use prepared statem|1     |#2270
> > >>>>>>>>>>>>>>> |bb56805e3801c7f8d49acd8095fd3099609d6fe5
> > >>>>>>>>>>>>>>> AIRFLOW-1168      |Bug         ||Major     ||Resolved
> > >>>> |DbApiHook
> > >>>>>>>>>>>> does
> > >>>>>>>>>>>>>>> not close connections when an excep|1     |#2269
> > >>>>>>>>>>>>>>> |777f181f81fa6a383a79b71fdcb9c4f4e77c785a
> > >>>>>>>>>>>>>>> AIRFLOW-1167      |Bug         ||Major     ||Resolved
> > >> |FTPHook
> > >>>>>> file
> > >>>>>>>>>>>>>>> modification date does not support op|1     |#2268
> > >>>>>>>>>>>>>>> |272c2f52142c167685646169431eb9dab85b7810
> > >>>>>>>>>>>>>>> AIRFLOW-1166      |Bug         ||Major     ||Resolved
> > >>>>>>>>>>>>>>> |_change_state_for_tis_without_dagrun can take very|1
> > >>>> |#2267
> > >>>>>>>>>>>>>>> |2389a8a7e89ea949e8f9674a3d3b18d2142dfece
> > >>>>>>>>>>>>>>> AIRFLOW-1160      |Bug         ||Major     ||Resolved
> > >> |Upadte
> > >>>>>>>>>>>>>>> SparkSubmitOperator parameters             |1     |#2265
> > >>>>>>>>>>>>>>> |69bd26913824c5c796a0a828a5c43ab39d0902f8
> > >>>>>>>>>>>>>>> AIRFLOW-1150      |Bug         ||Major     ||Resolved
> |Fix
> > >>>>>> scripts
> > >>>>>>>>>>>>>>> execution in SparkSql hook            |1     |#2259
> > >>>>>>>>>>>>>>> |093b2f04f7a010107fe6dfbc0658a53a1b8467e6
> > >>>>>>>>>>>>>>> AIRFLOW-1149      |Bug         ||Major     ||Resolved
> > |Allow
> > >>>>>> custom
> > >>>>>>>>>>>>>>> filters to be added to jinja2        |1     |#2258
> > >>>>>>>>>>>>>>> |9692510c4e067d0e81fd8d5893bfb4a0e53941e2
> > >>>>>>>>>>>>>>> AIRFLOW-1145      |Bug         ||Minor     ||Resolved
> > >>>>>>>>>>>>>>> |Closest_date_partition not working with before = T|1
> > >>>> |#2257
> > >>>>>>>>>>>>>>> |f446c0883cab30482436c47b867c4a440412f1ff
> > >>>>>>>>>>>>>>> AIRFLOW-1136      |Bug         ||Minor     ||Resolved
> > >> |Invalid
> > >>>>>>>>>>>>>>> parameters are not captured for Sqoop oper|0     |-
>  |-
> > >>>>>>>>>>>>>>> AIRFLOW-1119      |Bug         ||Major     ||Resolved
> > >>>> |Redshift
> > >>>>>> to
> > >>>>>>>>>>>> S3
> > >>>>>>>>>>>>>>> operator - headers not on first row|1     |#2245
> > >>>>>>>>>>>>>>> |6de5330d5336fb2455eea75c78e95f126320a5b0
> > >>>>>>>>>>>>>>> AIRFLOW-1089      |Improvement ||Major     ||Resolved
> |Add
> > >>>> Spark
> > >>>>>>>>>>>>>>> application arguments to SparkSubmitOper|1     |#2229
> > >>>>>>>>>>>>>>> |b4e9eb8310957cac47f6964662ce16496061db2c
> > >>>>>>>>>>>>>>> AIRFLOW-1081      |Bug         ||Major     ||Resolved
> > |Task
> > >>>>>>>>> duration
> > >>>>>>>>>>>>>>> page is slow                        |0     |-     |-
> > >>>>>>>>>>>>>>> AIRFLOW-1078      |Bug         ||Major     ||Resolved
> > >>>>>> |Latest_runs
> > >>>>>>>>>>>>>>> endpoint broken in old flask versions |1     |#2224
> > >>>>>>>>>>>>>>> |a4083f367b1f0f6525bcd011122d854626ea6768
> > >>>>>>>>>>>>>>> AIRFLOW-1074      |Bug         ||Major     ||Open
> |Do
> > >> not
> > >>>>>> count
> > >>>>>>>>>>>>>>> queued tasks in scheduler concurrency|1     |#2221
> > >>>>>>>>>>>>>>> |7a0284149a1dde4dc1d65a1d016eadf619e5320e
> > >>>>>>>>>>>>>>> AIRFLOW-1064      |Improvement ||Major     ||Resolved
> > >>>>>>>>>>>>>>> |TaskInstanceModelView is slow                     |1
> > >>>> |#2215
> > >>>>>>>>>>>>>>> |a2c18a58d0ba14101ddb70403fb95feda73cf21a
> > >>>>>>>>>>>>>>> AIRFLOW-1047      |Bug         ||Major     ||Resolved
> > >> |Airflow
> > >>>>>> logs
> > >>>>>>>>>>>>>>> vulnerable to XSS                    |0     |-     |-
> > >>>>>>>>>>>>>>> AIRFLOW-1041      |Bug         ||Major     ||Resolved
> > >>>>>>>>>>>> |DockerOperator
> > >>>>>>>>>>>>>>> replaces its xcom_push method with |0     |-     |-
> > >>>>>>>>>>>>>>> AIRFLOW-1038      |Bug         ||Major     ||Resolved
> > >> |Specify
> > >>>>>>>>>>>> celery
> > >>>>>>>>>>>>>>> serializers explicitly and pin vers|1     |#na
> > >>>>>>>>>>>>>>> |d1c64ab7b4ee6215900a15849e890ed1b408f5f5
> > >>>>>>>>>>>>>>> AIRFLOW-1036      |Improvement ||Major     ||Resolved
> > >>>>>> |Exponential
> > >>>>>>>>>>>>>>> backoff should use randomization      |1     |#2262
> > >>>>>>>>>>>>>>> |b4ee88aa6d71145daf91fc71763950ede16f8383
> > >>>>>>>>>>>>>>> AIRFLOW-1007      |Bug         ||Major     ||Resolved
> > |Jinja
> > >>>>>>>>> sandbox
> > >>>>>>>>>>>>>>> is vulnerable to RCE                |0     |-     |-
> > >>>>>>>>>>>>>>> AIRFLOW-993       |Bug         ||Minor     ||Resolved
> > |Dags
> > >>>>>> should
> > >>>>>>>>>>>>>>> modify the start date and end date of |1     |#2157
> > >>>>>>>>>>>>>>> |9fca409cebfd04ce8580c4657099a6db340eb7eb
> > >>>>>>>>>>>>>>> AIRFLOW-945       |Bug         ||Major     ||Resolved
> > >> |Revert
> > >>>>>>>>>>>>>>> psycopg2 workaround when psycopg2 2.7.1 is |0     |-
>  |-
> > >>>>>>>>>>>>>>> AIRFLOW-935       |Bug         ||Major     ||Resolved
> > >>>>>> |Impossible
> > >>>>>>>>> to
> > >>>>>>>>>>>>>>> use plugin executors                |0     |-     |-
> > >>>>>>>>>>>>>>> AIRFLOW-860       |Bug         ||Major     ||Resolved
> > >>>> |Circular
> > >>>>>>>>>>>>>>> module dependency prevents loading of cus|0     |-     |-
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Max
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> On Thu, Jun 8, 2017 at 12:18 AM, Bolke de Bruin <
> > >>>>>> bdbruin@gmail.com <mailto:bdbruin@gmail.com>>
> > >>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Hi Max,
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Sounds good. Couple of things:
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> * Can I suggest using the v1-8-test branch as the branch
> > to
> > >> be
> > >>>>>> used
> > >>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>> preparing the rc? If we hit RC then move it over to
> > >>>> v1-8-stable?
> > >>>>>>>>>>>>>>> V1-8-test
> > >>>>>>>>>>>>>>>> already had some fixes in that should land in 1.8.2 and
> > the
> > >> RC
> > >>>>>>>>>>>> should
> > >>>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>> tagged in the stable branch. That also reduces to amount
> > of
> > >>>>>> merge
> > >>>>>>>>>>>>>>> conflicts
> > >>>>>>>>>>>>>>>> probably as many have been merged. Where did you branch
> > off
> > >>>>>> from?
> > >>>>>>>>>>>>>>> Anyways,
> > >>>>>>>>>>>>>>>> see also the release management thing on the wiki.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Blocker(!)
> > >>>>>>>>>>>>>>>> * In the backfills we can loose tasks to execute due to
> a
> > >> task
> > >>>>>>>>>>>>> setting
> > >>>>>>>>>>>>>>> its
> > >>>>>>>>>>>>>>>> own state to NONE if concurrency limits are reached,
> this
> > >>>> makes
> > >>>>>>>>>>>> them
> > >>>>>>>>>>>>>> fall
> > >>>>>>>>>>>>>>>> outside of the scope the backfill is managing hence they
> > >> will
> > >>>>>> not
> > >>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>> executed (https://issues.apache.org/
> > >> jira/browse/AIRFLOW-1294
> > >>>> <
> > >>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1294> <
> > >>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1294 <
> > >>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1294>>). Setting
> > >>>>>>>>>>>> itself
> > >>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>> NONE should probably be “CONCURRENCY_REACHED” (new
> > state). I
> > >>>>>> have
> > >>>>>>>>>>>>>> marked
> > >>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>> as a blocker as we had multiple people hitting the
> issue,
> > >> but
> > >>>> I
> > >>>>>>>>>>>> need
> > >>>>>>>>>>>>>> 1-2
> > >>>>>>>>>>>>>>>> days to get a patch. Feel free to downgrade to critical
> if
> > >> you
> > >>>>>> like
> > >>>>>>>>>>>>> :).
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Cheers
> > >>>>>>>>>>>>>>>> Bolke
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> On 8 Jun 2017, at 02:35, Maxime Beauchemin <
> > >>>>>>>>>>>>>> maximebeauchemin@gmail.com <mailto:maximebeauchemin@
> > gmail.com
> > >>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> What a pleasant, mind numbing afternoon doing some
> > release
> > >>>>>>>>>>>>> management
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Notes:
> > >>>>>>>>>>>>>>>>> * Added a warning that the package name has changed on
> > Pypi
> > >>>>>>>>>>>>>>>>> <https://pypi.python.org/pypi/airflow <
> > >>>>>> https://pypi.python.org/pypi/airflow>>
> > >>>>>>>>>>>>>>>>> * Removed references to my name here
> > >>>>>>>>>>>>>>>>> <https://github.com/apache/incubator-airflow/pull/2352
> <
> > >>>>>> https://github.com/apache/incubator-airflow/pull/2352>> and
> > >>>>>>>>>>>> merged
> > >>>>>>>>>>>>>>>>> * Addressed John D. Ament's concerns here
> > >>>>>>>>>>>>>>>>> <https://github.com/apache/incubator-airflow/pull/2354
> <
> > >>>>>> https://github.com/apache/incubator-airflow/pull/2354>>, please
> > >>>>>>>>>>>>>>> review!
> > >>>>>>>>>>>>>>>>> * "footable" appears to have been removed, not a
> problem
> > >>>>>> anymore
> > >>>>>>>>>>>>>>>>> * that `airflow-jira is a god send! thanks Bolke.
> > >>>>>>>>>>>>>>>>> * reviewed list of Airbnb's production cherries and
> > flagged
> > >>>>>> those
> > >>>>>>>>>>>>> as
> > >>>>>>>>>>>>>>> `Fix
> > >>>>>>>>>>>>>>>>> Version == 1.8.2`
> > >>>>>>>>>>>>>>>>> * Started branch v1-8-2.rc1 and started picking
> cherries
> > >>>> using
> > >>>>>>>>>>>>>>>>> `airflow-jira compare 1.8.2`
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> I'll finish going through picking everything that
> > targeted
> > >>>>>> 1.8.2
> > >>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>> does
> > >>>>>>>>>>>>>>>>> not create merge conflict.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> If there's anything flagged as "blocker" that generates
> > >> merge
> > >>>>>>>>>>>>>> conflict,
> > >>>>>>>>>>>>>>>>> I'll go case by case about it.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Soon after, I should be able to announce 1.8.2 RC1,
> > >> hopefully
> > >>>>>>>>>>>>>> sometime
> > >>>>>>>>>>>>>>>>> tomorrow or Friday.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Let me know if there's anything else I'm missing that I
> > >>>> should
> > >>>>>>>>>>>>>>> consider.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Cheers!
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Max
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>
> > >>
> > >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message