Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 2CCA5200CB0 for ; Fri, 23 Jun 2017 19:55:18 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 2BB84160BE5; Fri, 23 Jun 2017 17:55:18 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id C990B160BE2 for ; Fri, 23 Jun 2017 19:55:16 +0200 (CEST) Received: (qmail 10883 invoked by uid 500); 23 Jun 2017 17:55:15 -0000 Mailing-List: contact dev-help@airflow.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@airflow.incubator.apache.org Delivered-To: mailing list dev@airflow.incubator.apache.org Received: (qmail 10871 invoked by uid 99); 23 Jun 2017 17:55:15 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 23 Jun 2017 17:55:15 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 1EADE1A0158 for ; Fri, 23 Jun 2017 17:55:15 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -0.651 X-Spam-Level: X-Spam-Status: No, score=-0.651 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, KAM_LOTSOFHASH=0.25, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-2.8, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id ONRVXb52Z4au for ; Fri, 23 Jun 2017 17:55:09 +0000 (UTC) Received: from mail-wr0-f173.google.com (mail-wr0-f173.google.com [209.85.128.173]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 630D35FC4D for ; Fri, 23 Jun 2017 17:55:08 +0000 (UTC) Received: by mail-wr0-f173.google.com with SMTP id c11so75424706wrc.3 for ; Fri, 23 Jun 2017 10:55:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=Gbktz3FJz2scNwbxNwVDI0+3y06xnQAZV7CHr95Afcw=; b=nyvy7NdsemfwbzzPxWENznzoBCG+nTQ3I4FHBIa82wFr7MOU9c3gwI6a+MBFS/Xs1J 2J0fWD5RNdSyP+VpwQwDiC4rZ51RucymrD+JE0RPVoOyDxWbxBx8FXudY2qeR6DkVoQY SKuF3eGSGDMTQqRyKUw3GStcCBH3lHXwKq7V5Mj0j+qQMUK0GUyKi57kBC5To3plS5tC nq/EDOVEazs4+EpGflcoCCzNeEdK2up5UpOwKUbxmIMI4NQdln2nbeMLWNdJPu0UI9WV Tq3F3p8GkQPX1TpXt0NDbw8h3fP/CXAMILX9A3ZOwisWI544lBAREkdSsuhgjc9tJjZp GZ2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=Gbktz3FJz2scNwbxNwVDI0+3y06xnQAZV7CHr95Afcw=; b=YKEVvm+gm8VAATjUK7ayv9I3MZRFaZIEC8J0VjOUnzZDUBdcNUR1qF59hM1bO4sTo1 XBfA9I9UAZv8UjpHW0JTalXx6u3nfxnKTA4lZcKZpEwSdQ/NOUQQDyAX9IJNOz9zfKKV odG/j6llTGc5ETKtwQNxHiGL8ktOWNUnyHUNz7Nj3fEO7cjblERiqyMcovEM7o0wEtX3 57AohshbWoFYPMu1qGyN3orND1o5TQmZXQ6c4mxjWjQoJz1jwYZnkWYt/BrK5/ZdpdFx f9fi44F3MGhNOOLFy2EsFHUoD/sF7HX4U1rZJ/bd5JuUARR8edh7WGZ0AvFTADt0YSaj i3VQ== X-Gm-Message-State: AKS2vOzOtoY9VzK91opOXozeGKbONZZwUzLVG0Q59zPoCbJXxJ3sR4v5 91v5ha5E7nfF56Y/Qfu/rylhlINZElY/ X-Received: by 10.28.95.214 with SMTP id t205mr6091029wmb.90.1498240506589; Fri, 23 Jun 2017 10:55:06 -0700 (PDT) MIME-Version: 1.0 Received: by 10.223.151.97 with HTTP; Fri, 23 Jun 2017 10:55:05 -0700 (PDT) In-Reply-To: References: <74FECE4A-054F-4385-A63F-769DFD9F2BCC@gmail.com> <03525365-BBF0-43B1-BBDF-9432B4A838D1@gmail.com> <58517917-EF93-4EB5-A6A5-F3E88DC7CAD5@gmail.com> <8D5EAACB-6487-4C1D-B200-D325F7353A7D@gmail.com> <05DAC1DE-5622-4AE1-85B1-F1C8C561E0D5@gmail.com> <70E94BA9-27ED-4921-861C-547014A6777E@gmail.com> <48803639-84ED-4C98-AEA2-DD007823E3E4@gmail.com> <6F8B6094-0513-444E-808D-2E01A06A9121@gmail.com> From: Maxime Beauchemin Date: Fri, 23 Jun 2017 10:55:05 -0700 Message-ID: Subject: Re: Release Manager for 1.8.2? To: dev@airflow.incubator.apache.org Content-Type: multipart/alternative; boundary="001a1148e4702efc1f0552a44d5f" archived-at: Fri, 23 Jun 2017 17:55:18 -0000 --001a1148e4702efc1f0552a44d5f Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable I appended my public key to the KEYS file. I'm not sure if I understand your 3rd point, should I remove all the previous tarballs and related files from https://dist.apache.org/repos/ dist/dev/incubator/airflow/ ? Max On Fri, Jun 23, 2017 at 5:48 AM, Bolke de Bruin wrote: > Hi Max, > > Looking on a phone, so I might not have spotted everything. > > - Tag looks good > - Your public GPG key has not been added yet to > https://dist.apache.org/repos/dist/release/incubator/airflow/KEYS < > https://dist.apache.org/repos/dist/release/incubator/airflow/KEYS> > - Please remove the old tar balls from https://dist.apache.org/repos/ > dist/dev/incubator/airflow/ repos/dist/dev/incubator/airflow/> is causes confusion for some reason at > the IPMC > > @chris please take a look as well. > > Bolke > > > On 23 Jun 2017, at 00:34, Maxime Beauchemin > wrote: > > > > Allllllllright, so I cherry-picked that last one, some of the other las= t > > minute ones had conflicts, and I bailed on all conflictual non-blocking > > issues. > > > > So here is the new tag, it's on the v1-8-test branch: > > https://github.com/apache/incubator-airflow/releases/tag/1.8.2rc2 > > > > The tarball, hashes and signature are here: > > https://dist.apache.org/repos/dist/dev/incubator/airflow/ > > > > Changelog: > > https://github.com/apache/incubator-airflow/blob/1.8.2rc2/CHANGELOG.txt > > > > Can someone confirm this looks alright before I send an announcement > about > > 1.8.2rc2 to the list? > > > > Max > > > > On Wed, Jun 21, 2017 at 10:44 PM, Bolke de Bruin > wrote: > > > >> Please also include https://github.com/apache/ > incubator-airflow/pull/2022 > >> > >> I have had reports it was also required for MySQL. > >> > >> Bolke > >> > >> Sent from my iPhone > >> > >>> On 22 Jun 2017, at 00:03, Maxime Beauchemin < > maximebeauchemin@gmail.com> > >> wrote: > >>> > >>> Ok I'll try to cherry-pick all of these, though if there are any merg= e > >>> conflicts I'll pass and move their Fix-Version to 1.9.0. > >>> > >>> Max > >>> > >>>> On Wed, Jun 21, 2017 at 1:20 AM, Bolke de Bruin > >> wrote: > >>>> > >>>> Hi max, > >>>> > >>>> PR 2365 is merged into master and also into v1-8-test. > >>>> > >>>> I have added: > >>>> > >>>> https://issues.apache.org/jira/browse/AIRFLOW-935 < > >>>> https://issues.apache.org/jira/browse/AIRFLOW-935> and > >>>> https://issues.apache.org/jira/browse/AIRFLOW-860 < > >>>> https://issues.apache.org/jira/browse/AIRFLOW-860> > >>>> > >>>> To the list fro 1.8.2. The PR for this has been in master for some > time: > >>>> https://github.com/apache/incubator-airflow/pull/2120 < > >>>> https://github.com/apache/incubator-airflow/pull/2120> > >>>> > >>>> Can you consider it for 1.8.2? Basically I promised to push to the > >> matter > >>>> :-). > >>>> > >>>> No blockers anymore as far as I know for 1.8.2. > >>>> > >>>> Bolke > >>>> > >>>> P.S. A friendly request if any of the committers do +1 or a LGTM to = a > >> PR, > >>>> please also merge the PR instead of leaving it to others. =E2=80=9Ca= irflow-pr > >> merge > >>>> XXX=E2=80=9D is there to help. > >>>> > >>>>> On 16 Jun 2017, at 15:11, Bolke de Bruin wrote: > >>>>> > >>>>> It is now pinned. I think Cloudera messed up their release for some > >>>> reason. > >>>>> > >>>>> Builds are succeeding again. > >>>>> > >>>>> Please note I have one blocker out for 1.8.2: > >> https://github.com/apache/ > >>>> incubator-airflow/pull/2365 >>>> incubator-airflow/pull/2365> > >>>>> > >>>>> Bolke > >>>>> > >>>>>> On 15 Jun 2017, at 16:43, Maxime Beauchemin < > >> maximebeauchemin@gmail.com > >>>> > wrote: > >>>>>> > >>>>>> Awesome, thanks for looking into it. Is there a way we can pin the > >>>> Cloudera > >>>>>> tarball to avoid surprises like this? > >>>>>> > >>>>>> On Thu, Jun 15, 2017 at 7:17 AM, Bolke de Bruin >>>> > wrote: > >>>>>> > >>>>>>> Found the issue for the permission denied part: beeline is not > >>>> executable > >>>>>>> in the latest tarball from Cloudera. I=E2=80=99ll have a workarou= nd for > this > >>>> in a > >>>>>>> few minutes and will push it to Apache right away. > >>>>>>> > >>>>>>> Bolke. > >>>>>>> > >>>>>>>> On 14 Jun 2017, at 18:15, Bolke de Bruin >> >>>> bdbruin@gmail.com>> wrote: > >>>>>>>> > >>>>>>>> Hi Max, > >>>>>>>> > >>>>>>>> I have created https://github.com/apache/ > >> incubator-airflow/pull/2365 > >>>> < > >>>>>>> https://github.com/apache/incubator-airflow/pull/2365 < > >>>> https://github.com/apache/incubator-airflow/pull/2365>> for > >> AIRFLOW-1296, > >>>>>>> which I think should be treated as a blocker. > >>>>>>>> > >>>>>>>> I think Travis=E2=80=99 is failing due to a dependency upgrade o= f Dask. > >>>>>>>> > >>>>>>>> The permission denied error seems to come from a new Travis conf= ig > >>>> that > >>>>>>> does not allow the creation of =E2=80=9C/user/hive/warehouse=E2= =80=9D by the normal > >>>> user. > >>>>>>> Probably a =E2=80=9Csudo mkdir =E2=80=9C and a =E2=80=9Csudo chow= n=E2=80=9D will help here. Let me > >>>> check. > >>>>>>>> > >>>>>>>> Bolke > >>>>>>>> > >>>>>>>>> On 14 Jun 2017, at 00:26, Maxime Beauchemin < > >>>> maximebeauchemin@gmail.com > >>>>>>> >> gmail.com>>> > >>>> wrote: > >>>>>>>>> > >>>>>>>>> A quick update on my progress. > >>>>>>>>> > >>>>>>>>> I ran through the "Releasing Airflow" wiki playbook only to > >> realize I > >>>>>>> had > >>>>>>>>> published a version that failed the Travis build > >>>>>>>>> >> airflow/jobs/242532965 > >>>> > < > >>>>>>> https://travis-ci.org/mistercrunch/incubator- > airflow/jobs/242532965 > >> < > >>>> https://travis-ci.org/mistercrunch/incubator-airflow/jobs/242532965 > >>> > >>>>>>>>> afterwards. I've > >>>>>>>>> been updating the wiki as I go and it seems like it will be > better > >>>> the > >>>>>>> next > >>>>>>>>> time around. I'm getting "Permission denied" on the Hive hook > >>>>>>> subprocess if > >>>>>>>>> anyone can shed some light on that. > >>>>>>>>> > >>>>>>>>> So I need to fix the unit tests at this point, and probably go > >>>> straight > >>>>>>> to > >>>>>>>>> `rc2` to avoid changing the files out there for `rc1`. > >>>>>>>>> > >>>>>>>>> Max > >>>>>>>>> > >>>>>>>>> On Thu, Jun 8, 2017 at 4:54 PM, Ruslan Dautkhanov < > >>>> dautkhanov@gmail.com > >>>>>>> >> > >>>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>>> It would be great if somebody would have a look at following 3 > >> jiras > >>>>>>>>>> > >>>>>>>>>> I've flagged > >>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013 < > >>>> https://issues.apache.org/jira/browse/AIRFLOW-1013> < > >>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013 < > >>>> https://issues.apache.org/jira/browse/AIRFLOW-1013>> > >>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1178 < > >>>> https://issues.apache.org/jira/browse/AIRFLOW-1178> > >>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1055 > >>>>>>>>>> > >>>>>>>>>> Two of them were Blockers for one of previous 1.8 releases but > >> were > >>>>>>>>>> de-escalated because no resources to fix them. > >>>>>>>>>> > >>>>>>>>>> I'm currently Assignee on AIRFLOW-1055 - can't unassign myself= - > >>>> feel > >>>>>>> free > >>>>>>>>>> to scratch my name there. > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> -- > >>>>>>>>>> Ruslan Dautkhanov > >>>>>>>>>> > >>>>>>>>>> On Fri, Jun 9, 2017 at 12:23 AM, Maxime Beauchemin < > >>>>>>>>>> maximebeauchemin@gmail.com = > > >>>> wrote: > >>>>>>>>>> > >>>>>>>>>>> Cool. I'll let everyone know when RC1 is out. > >>>>>>>>>>> > >>>>>>>>>>> @community, if you think anything else should be a blocker fo= r > >>>> 1.8.2 > >>>>>>>>>> please > >>>>>>>>>>> flag it in Jira very soon or it will miss the release train! > >>>>>>>>>>> > >>>>>>>>>>> Max > >>>>>>>>>>> > >>>>>>>>>>> On Thu, Jun 8, 2017 at 9:06 AM, Chris Riccomini < > >>>>>>> criccomini@apache.org > > >>>>>>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>>> Thanks for doing all this, Max! When you're ready, I can > deploy > >> to > >>>>>>> some > >>>>>>>>>>> of > >>>>>>>>>>>> our environments to verify. Just let me know. > >>>>>>>>>>>> > >>>>>>>>>>>> On Thu, Jun 8, 2017 at 8:42 AM, Maxime Beauchemin < > >>>>>>>>>>>> maximebeauchemin@gmail.com >> > >>>> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>>> I branched off `v1-8-test` so all should be good. I just > didn't > >>>> know > >>>>>>>>>>> if I > >>>>>>>>>>>>> could move ahead with that branch just yet so I branched > off. I > >>>> just > >>>>>>>>>>> got > >>>>>>>>>>>>> back on `v1-8-test` and pushed what I have to Apache > [somehow I > >>>> had > >>>>>>>>>> to > >>>>>>>>>>>>> rebase meaning someone added something over the past 24 > hours]. > >>>>>>>>>>>>> > >>>>>>>>>>>>> I just set `Fix Version` of AIRFLOW-1294 to 1.8.2 and set i= t > to > >>>>>>>>>>> blocker. > >>>>>>>>>>>>> > >>>>>>>>>>>>> I cherry-picked all the commits targeting 1.8.2 I could > without > >>>>>>>>>> getting > >>>>>>>>>>>>> into conflicts. I'd only work at resolving conflicts on > >>>> blockers. My > >>>>>>>>>>> plan > >>>>>>>>>>>>> is to add only the 2 rows in red for RC1. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Here's the current output of `airflow-jira 1.8.2`: > >>>>>>>>>>>>> > >>>>>>>>>>>>> ISSUE ID |TYPE ||PRIORITY ||STATUS > >>>> |DESCRIPTION > >>>>>>>>>>>>> |MERGED|PR > >>>> |COMMITAIRFLOW-1294 > >>>>>>>>>>>>> |Bug ||Blocker ||Open |Backfills can loose > tasks > >>>> to > >>>>>>>>>>>>> execute due to tasks |0 |- |- > >>>>>>>>>>>>> AIRFLOW-1291 |Bug ||Blocker ||Open > >> |Update > >>>>>>>>>>>>> NOTICE and LICENSE files to meet ASF specs |0 |- |- > >>>>>>>>>>>>> AIRFLOW-1290 |Bug ||Major ||Resolved > |Change > >>>> docs > >>>>>>>>>>>>> author from "Maxime Beauchemin" to "Ap|1 |#na > >>>>>>>>>>>>> |fc33c040605e7b0121739af841790336bdfa3948 > >>>>>>>>>>>>> AIRFLOW-1282 |Bug ||Major ||Resolved |Fix > >> known > >>>>>>>>>>>>> event column sorting |1 |#2350 > >>>>>>>>>>>>> |2e9eee3697f6261de61c77ba584ace14b22751f0 > >>>>>>>>>>>>> AIRFLOW-1281 |Bug ||Minor ||Resolved |Sor= t > >>>>>>>>>> variables > >>>>>>>>>>>>> by key field by default |1 |#2347 > >>>>>>>>>>>>> |57d5bcdad4202fd8a94be07b4aeacee065801481 > >>>>>>>>>>>>> AIRFLOW-1275 |Bug ||Major ||In Progress|Fi= x > >>>>>>> `airflow > >>>>>>>>>>>>> pool` command exception |0 |- |- > >>>>>>>>>>>>> AIRFLOW-1274 |Bug ||Major ||Open > >>>> |HttpSensor > >>>>>>>>>>>>> parameter params is overriding BaseOper|0 |- |- > >>>>>>>>>>>>> AIRFLOW-1266 |Bug ||Major ||Open |Lon= g > >> task > >>>>>>>>>>>>> names are truncated in gannt view |1 |#2345 > >>>>>>>>>>>>> |7f33f6e2fe4149e547860fdb237da26a5b7023ec > >>>>>>>>>>>>> AIRFLOW-1263 |Bug ||Major ||Resolved > |Airflow > >>>>>>>>>> Charts > >>>>>>>>>>>>> Pages Should Have Dynamic Heights |1 |#2344 > >>>>>>>>>>>>> |acd0166d8c7f5639898791848d985f6441b3b5b5 > >>>>>>>>>>>>> AIRFLOW-1244 |Bug ||Major ||Resolved > |Forbid > >>>>>>>>>>>>> creation of a pool with empty name |1 |#2324 > >>>>>>>>>>>>> |802fc1549d0f173c646794f2476156d34383adc0 > >>>>>>>>>>>>> AIRFLOW-1243 |Bug ||Minor ||Resolved |DAG= s > >>>> table > >>>>>>>>>> has > >>>>>>>>>>>>> no default entries to show |1 |#2323 > >>>>>>>>>>>>> |1232b6a4fd691e0faf081d4cf792a6d96d33b595 > >>>>>>>>>>>>> AIRFLOW-1227 |Bug ||Minor ||Resolved > |Remove > >>>> empty > >>>>>>>>>>>>> column on the Logs view |1 |#2310 > >>>>>>>>>>>>> |b0ba3c91044b2abd92289d9b3f7d913060ef6c31 > >>>>>>>>>>>>> AIRFLOW-1226 |Bug ||Minor ||Resolved > |Remove > >>>> empty > >>>>>>>>>>>>> column on the Jobs view |1 |#2309 > >>>>>>>>>>>>> |c406652bd4c8f8099841c69f2d1ccc9678d77bfd > >>>>>>>>>>>>> AIRFLOW-1221 |Bug ||Major ||Resolved |Fix > >>>>>>>>>>>>> DatabricksSubmitRunOperator Templating |0 |- > |- > >>>>>>>>>>>>> AIRFLOW-1217 |Bug ||Major ||Resolved > |Enable > >>>>>>>>>> logging > >>>>>>>>>>>>> in Sqoop hook |0 |- |- > >>>>>>>>>>>>> AIRFLOW-1213 |Bug ||Major ||Resolved |Add > >>>> hcatalog > >>>>>>>>>>>>> parameters to the sqoop operator/hook|0 |- |- > >>>>>>>>>>>>> AIRFLOW-1203 |Bug ||Major ||Closed |Tes= ts > >>>>>>> failing > >>>>>>>>>>>>> after oauth upgrade |0 |- |- > >>>>>>>>>>>>> AIRFLOW-1201 |Bug ||Major ||Resolved > |Update > >>>>>>>>>>>>> deprecated 'nose-parameterized' library to |0 |- |- > >>>>>>>>>>>>> AIRFLOW-1200 |Bug ||Major ||Resolved > |Forbid > >>>>>>>>>>>>> creation of a variable with an empty key |1 |#2299 > >>>>>>>>>>>>> |cac7d4cb571a4c1f7ba53360765ea6c640c719df > >>>>>>>>>>>>> AIRFLOW-1199 |Bug ||Major ||Resolved > |Create > >>>> modal > >>>>>>>>>>>>> looks like edit modal (edit tab selec|1 |#2293 > >>>>>>>>>>>>> |51a83cce96d41eb1b9df9f82188c36c7e12061df > >>>>>>>>>>>>> AIRFLOW-1197 |Bug ||Major ||Open > |Contrib > >>>>>>> Spark > >>>>>>>>>>>>> Submit Hook on_kill error |0 |- |- > >>>>>>>>>>>>> AIRFLOW-1192 |New Feature ||Major ||Resolved |Add > >>>> support > >>>>>>>>>> to > >>>>>>>>>>>>> run Zeppelin Notebooks with Qubole |1 |#2322 > >>>>>>>>>>>>> |bf966e63be2deb19fa2824efdfe854736318f1b5 > >>>>>>>>>>>>> AIRFLOW-1191 |Improvement ||Major ||Open > |Contrib > >>>>>>> Spark > >>>>>>>>>>>>> Submit hook should permit override o|0 |- |- > >>>>>>>>>>>>> AIRFLOW-1186 |Bug ||Major ||Resolved |Tas= k > >>>>>>> duration > >>>>>>>>>>>>> shows weird graph for some dags |1 |#2284 > >>>>>>>>>>>>> |5f3ee52da7ca4f1fca8a84b3281b31aa6241514e > >>>>>>>>>>>>> AIRFLOW-1180 |Bug ||Major ||Resolved |Fix > >>>>>>> flask-wtf > >>>>>>>>>>>>> version in setup.py for test_csrf_re|1 |#2280 > >>>>>>>>>>>>> |93b8e9643cc273ec0ecc063b9f851d9a8012cd03 > >>>>>>>>>>>>> AIRFLOW-1179 |Bug ||Major ||Closed > |Pandas > >>>> 0.20 > >>>>>>>>>>>>> broke Google BigQuery hook |1 |#2279 > >>>>>>>>>>>>> |c7c0b72d934fec10d890f7a3bd8fd06c0e2f44bd > >>>>>>>>>>>>> AIRFLOW-1170 |Bug ||Major ||Resolved > >> |DbApiHook > >>>>>>>>>>>>> insert rows does not use prepared statem|1 |#2270 > >>>>>>>>>>>>> |bb56805e3801c7f8d49acd8095fd3099609d6fe5 > >>>>>>>>>>>>> AIRFLOW-1168 |Bug ||Major ||Resolved > >> |DbApiHook > >>>>>>>>>> does > >>>>>>>>>>>>> not close connections when an excep|1 |#2269 > >>>>>>>>>>>>> |777f181f81fa6a383a79b71fdcb9c4f4e77c785a > >>>>>>>>>>>>> AIRFLOW-1167 |Bug ||Major ||Resolved > |FTPHook > >>>> file > >>>>>>>>>>>>> modification date does not support op|1 |#2268 > >>>>>>>>>>>>> |272c2f52142c167685646169431eb9dab85b7810 > >>>>>>>>>>>>> AIRFLOW-1166 |Bug ||Major ||Resolved > >>>>>>>>>>>>> |_change_state_for_tis_without_dagrun can take very|1 > >> |#2267 > >>>>>>>>>>>>> |2389a8a7e89ea949e8f9674a3d3b18d2142dfece > >>>>>>>>>>>>> AIRFLOW-1160 |Bug ||Major ||Resolved > |Upadte > >>>>>>>>>>>>> SparkSubmitOperator parameters |1 |#2265 > >>>>>>>>>>>>> |69bd26913824c5c796a0a828a5c43ab39d0902f8 > >>>>>>>>>>>>> AIRFLOW-1150 |Bug ||Major ||Resolved |Fix > >>>> scripts > >>>>>>>>>>>>> execution in SparkSql hook |1 |#2259 > >>>>>>>>>>>>> |093b2f04f7a010107fe6dfbc0658a53a1b8467e6 > >>>>>>>>>>>>> AIRFLOW-1149 |Bug ||Major ||Resolved |All= ow > >>>> custom > >>>>>>>>>>>>> filters to be added to jinja2 |1 |#2258 > >>>>>>>>>>>>> |9692510c4e067d0e81fd8d5893bfb4a0e53941e2 > >>>>>>>>>>>>> AIRFLOW-1145 |Bug ||Minor ||Resolved > >>>>>>>>>>>>> |Closest_date_partition not working with before =3D T|1 > >> |#2257 > >>>>>>>>>>>>> |f446c0883cab30482436c47b867c4a440412f1ff > >>>>>>>>>>>>> AIRFLOW-1136 |Bug ||Minor ||Resolved > |Invalid > >>>>>>>>>>>>> parameters are not captured for Sqoop oper|0 |- |- > >>>>>>>>>>>>> AIRFLOW-1119 |Bug ||Major ||Resolved > >> |Redshift > >>>> to > >>>>>>>>>> S3 > >>>>>>>>>>>>> operator - headers not on first row|1 |#2245 > >>>>>>>>>>>>> |6de5330d5336fb2455eea75c78e95f126320a5b0 > >>>>>>>>>>>>> AIRFLOW-1089 |Improvement ||Major ||Resolved |Add > >> Spark > >>>>>>>>>>>>> application arguments to SparkSubmitOper|1 |#2229 > >>>>>>>>>>>>> |b4e9eb8310957cac47f6964662ce16496061db2c > >>>>>>>>>>>>> AIRFLOW-1081 |Bug ||Major ||Resolved |Tas= k > >>>>>>> duration > >>>>>>>>>>>>> page is slow |0 |- |- > >>>>>>>>>>>>> AIRFLOW-1078 |Bug ||Major ||Resolved > >>>> |Latest_runs > >>>>>>>>>>>>> endpoint broken in old flask versions |1 |#2224 > >>>>>>>>>>>>> |a4083f367b1f0f6525bcd011122d854626ea6768 > >>>>>>>>>>>>> AIRFLOW-1074 |Bug ||Major ||Open |Do > not > >>>> count > >>>>>>>>>>>>> queued tasks in scheduler concurrency|1 |#2221 > >>>>>>>>>>>>> |7a0284149a1dde4dc1d65a1d016eadf619e5320e > >>>>>>>>>>>>> AIRFLOW-1064 |Improvement ||Major ||Resolved > >>>>>>>>>>>>> |TaskInstanceModelView is slow |1 > >> |#2215 > >>>>>>>>>>>>> |a2c18a58d0ba14101ddb70403fb95feda73cf21a > >>>>>>>>>>>>> AIRFLOW-1047 |Bug ||Major ||Resolved > |Airflow > >>>> logs > >>>>>>>>>>>>> vulnerable to XSS |0 |- |- > >>>>>>>>>>>>> AIRFLOW-1041 |Bug ||Major ||Resolved > >>>>>>>>>> |DockerOperator > >>>>>>>>>>>>> replaces its xcom_push method with |0 |- |- > >>>>>>>>>>>>> AIRFLOW-1038 |Bug ||Major ||Resolved > |Specify > >>>>>>>>>> celery > >>>>>>>>>>>>> serializers explicitly and pin vers|1 |#na > >>>>>>>>>>>>> |d1c64ab7b4ee6215900a15849e890ed1b408f5f5 > >>>>>>>>>>>>> AIRFLOW-1036 |Improvement ||Major ||Resolved > >>>> |Exponential > >>>>>>>>>>>>> backoff should use randomization |1 |#2262 > >>>>>>>>>>>>> |b4ee88aa6d71145daf91fc71763950ede16f8383 > >>>>>>>>>>>>> AIRFLOW-1007 |Bug ||Major ||Resolved |Jin= ja > >>>>>>> sandbox > >>>>>>>>>>>>> is vulnerable to RCE |0 |- |- > >>>>>>>>>>>>> AIRFLOW-993 |Bug ||Minor ||Resolved |Dag= s > >>>> should > >>>>>>>>>>>>> modify the start date and end date of |1 |#2157 > >>>>>>>>>>>>> |9fca409cebfd04ce8580c4657099a6db340eb7eb > >>>>>>>>>>>>> AIRFLOW-945 |Bug ||Major ||Resolved > |Revert > >>>>>>>>>>>>> psycopg2 workaround when psycopg2 2.7.1 is |0 |- |- > >>>>>>>>>>>>> AIRFLOW-935 |Bug ||Major ||Resolved > >>>> |Impossible > >>>>>>> to > >>>>>>>>>>>>> use plugin executors |0 |- |- > >>>>>>>>>>>>> AIRFLOW-860 |Bug ||Major ||Resolved > >> |Circular > >>>>>>>>>>>>> module dependency prevents loading of cus|0 |- |- > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> Max > >>>>>>>>>>>>> > >>>>>>>>>>>>> On Thu, Jun 8, 2017 at 12:18 AM, Bolke de Bruin < > >>>> bdbruin@gmail.com > > >>>>>>>>>>>> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>>> Hi Max, > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Sounds good. Couple of things: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> * Can I suggest using the v1-8-test branch as the branch t= o > be > >>>> used > >>>>>>>>>>> for > >>>>>>>>>>>>>> preparing the rc? If we hit RC then move it over to > >> v1-8-stable? > >>>>>>>>>>>>> V1-8-test > >>>>>>>>>>>>>> already had some fixes in that should land in 1.8.2 and th= e > RC > >>>>>>>>>> should > >>>>>>>>>>>> be > >>>>>>>>>>>>>> tagged in the stable branch. That also reduces to amount o= f > >>>> merge > >>>>>>>>>>>>> conflicts > >>>>>>>>>>>>>> probably as many have been merged. Where did you branch of= f > >>>> from? > >>>>>>>>>>>>> Anyways, > >>>>>>>>>>>>>> see also the release management thing on the wiki. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Blocker(!) > >>>>>>>>>>>>>> * In the backfills we can loose tasks to execute due to a > task > >>>>>>>>>>> setting > >>>>>>>>>>>>> its > >>>>>>>>>>>>>> own state to NONE if concurrency limits are reached, this > >> makes > >>>>>>>>>> them > >>>>>>>>>>>> fall > >>>>>>>>>>>>>> outside of the scope the backfill is managing hence they > will > >>>> not > >>>>>>>>>> be > >>>>>>>>>>>>>> executed (https://issues.apache.org/ > jira/browse/AIRFLOW-1294 > >> < > >>>> https://issues.apache.org/jira/browse/AIRFLOW-1294> < > >>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1294 < > >>>> https://issues.apache.org/jira/browse/AIRFLOW-1294>>). Setting > >>>>>>>>>> itself > >>>>>>>>>>>> to > >>>>>>>>>>>>>> NONE should probably be =E2=80=9CCONCURRENCY_REACHED=E2=80= =9D (new state). I > >>>> have > >>>>>>>>>>>> marked > >>>>>>>>>>>>> it > >>>>>>>>>>>>>> as a blocker as we had multiple people hitting the issue, > but > >> I > >>>>>>>>>> need > >>>>>>>>>>>> 1-2 > >>>>>>>>>>>>>> days to get a patch. Feel free to downgrade to critical if > you > >>>> like > >>>>>>>>>>> :). > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Cheers > >>>>>>>>>>>>>> Bolke > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> On 8 Jun 2017, at 02:35, Maxime Beauchemin < > >>>>>>>>>>>> maximebeauchemin@gmail.com > > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> What a pleasant, mind numbing afternoon doing some releas= e > >>>>>>>>>>> management > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Notes: > >>>>>>>>>>>>>>> * Added a warning that the package name has changed on Py= pi > >>>>>>>>>>>>>>> >>>> https://pypi.python.org/pypi/airflow>> > >>>>>>>>>>>>>>> * Removed references to my name here > >>>>>>>>>>>>>>> >>>> https://github.com/apache/incubator-airflow/pull/2352>> and > >>>>>>>>>> merged > >>>>>>>>>>>>>>> * Addressed John D. Ament's concerns here > >>>>>>>>>>>>>>> >>>> https://github.com/apache/incubator-airflow/pull/2354>>, please > >>>>>>>>>>>>> review! > >>>>>>>>>>>>>>> * "footable" appears to have been removed, not a problem > >>>> anymore > >>>>>>>>>>>>>>> * that `airflow-jira is a god send! thanks Bolke. > >>>>>>>>>>>>>>> * reviewed list of Airbnb's production cherries and flagg= ed > >>>> those > >>>>>>>>>>> as > >>>>>>>>>>>>> `Fix > >>>>>>>>>>>>>>> Version =3D=3D 1.8.2` > >>>>>>>>>>>>>>> * Started branch v1-8-2.rc1 and started picking cherries > >> using > >>>>>>>>>>>>>>> `airflow-jira compare 1.8.2` > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> I'll finish going through picking everything that targete= d > >>>> 1.8.2 > >>>>>>>>>>> that > >>>>>>>>>>>>>> does > >>>>>>>>>>>>>>> not create merge conflict. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> If there's anything flagged as "blocker" that generates > merge > >>>>>>>>>>>> conflict, > >>>>>>>>>>>>>>> I'll go case by case about it. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Soon after, I should be able to announce 1.8.2 RC1, > hopefully > >>>>>>>>>>>> sometime > >>>>>>>>>>>>>>> tomorrow or Friday. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Let me know if there's anything else I'm missing that I > >> should > >>>>>>>>>>>>> consider. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Cheers! > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Max > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>>> > >>>>> > >>>> > >>>> > >> > > --001a1148e4702efc1f0552a44d5f--