airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maxime Beauchemin <maximebeauche...@gmail.com>
Subject Re: Release Manager for 1.8.2?
Date Fri, 23 Jun 2017 17:55:05 GMT
I appended my public key to the KEYS file.

I'm not sure if I understand your 3rd point, should I remove all the
previous tarballs and related files from https://dist.apache.org/repos/
dist/dev/incubator/airflow/ ?

Max

On Fri, Jun 23, 2017 at 5:48 AM, Bolke de Bruin <bdbruin@gmail.com> wrote:

> Hi Max,
>
> Looking on a phone, so I might not have spotted everything.
>
> - Tag looks good
> - Your public GPG key has not been added yet to
> https://dist.apache.org/repos/dist/release/incubator/airflow/KEYS <
> https://dist.apache.org/repos/dist/release/incubator/airflow/KEYS>
> - Please remove the old tar balls from https://dist.apache.org/repos/
> dist/dev/incubator/airflow/ <https://dist.apache.org/
> repos/dist/dev/incubator/airflow/> is causes confusion for some reason at
> the IPMC
>
> @chris please take a look as well.
>
> Bolke
>
> > On 23 Jun 2017, at 00:34, Maxime Beauchemin <maximebeauchemin@gmail.com>
> wrote:
> >
> > Allllllllright, so I cherry-picked that last one, some of the other last
> > minute ones had conflicts, and I bailed on all conflictual non-blocking
> > issues.
> >
> > So here is the new tag, it's on the v1-8-test branch:
> > https://github.com/apache/incubator-airflow/releases/tag/1.8.2rc2
> >
> > The tarball, hashes and signature are here:
> > https://dist.apache.org/repos/dist/dev/incubator/airflow/
> >
> > Changelog:
> > https://github.com/apache/incubator-airflow/blob/1.8.2rc2/CHANGELOG.txt
> >
> > Can someone confirm this looks alright before I send an announcement
> about
> > 1.8.2rc2 to the list?
> >
> > Max
> >
> > On Wed, Jun 21, 2017 at 10:44 PM, Bolke de Bruin <bdbruin@gmail.com>
> wrote:
> >
> >> Please also include https://github.com/apache/
> incubator-airflow/pull/2022
> >>
> >> I have had reports it was also required for MySQL.
> >>
> >> Bolke
> >>
> >> Sent from my iPhone
> >>
> >>> On 22 Jun 2017, at 00:03, Maxime Beauchemin <
> maximebeauchemin@gmail.com>
> >> wrote:
> >>>
> >>> Ok I'll try to cherry-pick all of these, though if there are any merge
> >>> conflicts I'll pass and move their Fix-Version to 1.9.0.
> >>>
> >>> Max
> >>>
> >>>> On Wed, Jun 21, 2017 at 1:20 AM, Bolke de Bruin <bdbruin@gmail.com>
> >> wrote:
> >>>>
> >>>> Hi max,
> >>>>
> >>>> PR 2365 is merged into master and also into v1-8-test.
> >>>>
> >>>> I have added:
> >>>>
> >>>> https://issues.apache.org/jira/browse/AIRFLOW-935 <
> >>>> https://issues.apache.org/jira/browse/AIRFLOW-935> and
> >>>> https://issues.apache.org/jira/browse/AIRFLOW-860 <
> >>>> https://issues.apache.org/jira/browse/AIRFLOW-860>
> >>>>
> >>>> To the list fro 1.8.2. The PR for this has been in master for some
> time:
> >>>> https://github.com/apache/incubator-airflow/pull/2120 <
> >>>> https://github.com/apache/incubator-airflow/pull/2120>
> >>>>
> >>>> Can you consider it for 1.8.2? Basically I promised to push to the
> >> matter
> >>>> :-).
> >>>>
> >>>> No blockers anymore as far as I know for 1.8.2.
> >>>>
> >>>> Bolke
> >>>>
> >>>> P.S. A friendly request if any of the committers do +1 or a LGTM to
a
> >> PR,
> >>>> please also merge the PR instead of leaving it to others. “airflow-pr
> >> merge
> >>>> XXX” is there to help.
> >>>>
> >>>>> On 16 Jun 2017, at 15:11, Bolke de Bruin <bdbruin@gmail.com>
wrote:
> >>>>>
> >>>>> It is now pinned. I think Cloudera messed up their release for some
> >>>> reason.
> >>>>>
> >>>>> Builds are succeeding again.
> >>>>>
> >>>>> Please note I have one blocker out for 1.8.2:
> >> https://github.com/apache/
> >>>> incubator-airflow/pull/2365 <https://github.com/apache/
> >>>> incubator-airflow/pull/2365>
> >>>>>
> >>>>> Bolke
> >>>>>
> >>>>>> On 15 Jun 2017, at 16:43, Maxime Beauchemin <
> >> maximebeauchemin@gmail.com
> >>>> <mailto:maximebeauchemin@gmail.com>> wrote:
> >>>>>>
> >>>>>> Awesome, thanks for looking into it. Is there a way we can pin
the
> >>>> Cloudera
> >>>>>> tarball to avoid surprises like this?
> >>>>>>
> >>>>>> On Thu, Jun 15, 2017 at 7:17 AM, Bolke de Bruin <bdbruin@gmail.com
> >>>> <mailto:bdbruin@gmail.com>> wrote:
> >>>>>>
> >>>>>>> Found the issue for the permission denied part: beeline
is not
> >>>> executable
> >>>>>>> in the latest tarball from Cloudera. I’ll have a workaround
for
> this
> >>>> in a
> >>>>>>> few minutes and will push it to Apache right away.
> >>>>>>>
> >>>>>>> Bolke.
> >>>>>>>
> >>>>>>>> On 14 Jun 2017, at 18:15, Bolke de Bruin <bdbruin@gmail.com
> >> <mailto:
> >>>> bdbruin@gmail.com>> wrote:
> >>>>>>>>
> >>>>>>>> Hi Max,
> >>>>>>>>
> >>>>>>>> I have created https://github.com/apache/
> >> incubator-airflow/pull/2365
> >>>> <https://github.com/apache/incubator-airflow/pull/2365> <
> >>>>>>> https://github.com/apache/incubator-airflow/pull/2365 <
> >>>> https://github.com/apache/incubator-airflow/pull/2365>> for
> >> AIRFLOW-1296,
> >>>>>>> which I think should be treated as a blocker.
> >>>>>>>>
> >>>>>>>> I think Travis’ is failing due to a dependency upgrade
of Dask.
> >>>>>>>>
> >>>>>>>> The permission denied error seems to come from a new
Travis config
> >>>> that
> >>>>>>> does not allow the creation of “/user/hive/warehouse”
by the normal
> >>>> user.
> >>>>>>> Probably a “sudo mkdir “ and a “sudo chown” will
help here. Let me
> >>>> check.
> >>>>>>>>
> >>>>>>>> Bolke
> >>>>>>>>
> >>>>>>>>> On 14 Jun 2017, at 00:26, Maxime Beauchemin <
> >>>> maximebeauchemin@gmail.com <mailto:maximebeauchemin@gmail.com>
> >>>>>>> <mailto:maximebeauchemin@gmail.com <mailto:maximebeauchemin@
> >> gmail.com>>>
> >>>> wrote:
> >>>>>>>>>
> >>>>>>>>> A quick update on my progress.
> >>>>>>>>>
> >>>>>>>>> I ran through the "Releasing Airflow" wiki playbook
only to
> >> realize I
> >>>>>>> had
> >>>>>>>>> published a version that failed the Travis build
> >>>>>>>>> <https://travis-ci.org/mistercrunch/incubator-
> >> airflow/jobs/242532965
> >>>> <https://travis-ci.org/mistercrunch/incubator-airflow/jobs/242532965>
> <
> >>>>>>> https://travis-ci.org/mistercrunch/incubator-
> airflow/jobs/242532965
> >> <
> >>>> https://travis-ci.org/mistercrunch/incubator-airflow/jobs/242532965
> >>>
> >>>>>>>>> afterwards. I've
> >>>>>>>>> been updating the wiki as I go and it seems like
it will be
> better
> >>>> the
> >>>>>>> next
> >>>>>>>>> time around. I'm getting "Permission denied" on
the Hive hook
> >>>>>>> subprocess if
> >>>>>>>>> anyone can shed some light on that.
> >>>>>>>>>
> >>>>>>>>> So I need to fix the unit tests at this point, and
probably go
> >>>> straight
> >>>>>>> to
> >>>>>>>>> `rc2` to avoid changing the files out there for
`rc1`.
> >>>>>>>>>
> >>>>>>>>> Max
> >>>>>>>>>
> >>>>>>>>> On Thu, Jun 8, 2017 at 4:54 PM, Ruslan Dautkhanov
<
> >>>> dautkhanov@gmail.com <mailto:dautkhanov@gmail.com>
> >>>>>>> <mailto:dautkhanov@gmail.com <mailto:dautkhanov@gmail.com>>>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> It would be great if somebody would have a look
at following 3
> >> jiras
> >>>>>>>>>>
> >>>>>>>>>> I've flagged
> >>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013
<
> >>>> https://issues.apache.org/jira/browse/AIRFLOW-1013> <
> >>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013 <
> >>>> https://issues.apache.org/jira/browse/AIRFLOW-1013>>
> >>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1178
<
> >>>> https://issues.apache.org/jira/browse/AIRFLOW-1178>
> >>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1055
> >>>>>>>>>>
> >>>>>>>>>> Two of them were Blockers for one of previous
1.8 releases but
> >> were
> >>>>>>>>>> de-escalated because no resources to fix them.
> >>>>>>>>>>
> >>>>>>>>>> I'm currently Assignee on AIRFLOW-1055 - can't
unassign myself -
> >>>> feel
> >>>>>>> free
> >>>>>>>>>> to scratch my name there.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> Ruslan Dautkhanov
> >>>>>>>>>>
> >>>>>>>>>> On Fri, Jun 9, 2017 at 12:23 AM, Maxime Beauchemin
<
> >>>>>>>>>> maximebeauchemin@gmail.com <mailto:maximebeauchemin@gmail.com>>
> >>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Cool. I'll let everyone know when RC1 is
out.
> >>>>>>>>>>>
> >>>>>>>>>>> @community, if you think anything else should
be a blocker for
> >>>> 1.8.2
> >>>>>>>>>> please
> >>>>>>>>>>> flag it in Jira very soon or it will miss
the release train!
> >>>>>>>>>>>
> >>>>>>>>>>> Max
> >>>>>>>>>>>
> >>>>>>>>>>> On Thu, Jun 8, 2017 at 9:06 AM, Chris Riccomini
<
> >>>>>>> criccomini@apache.org <mailto:criccomini@apache.org>>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Thanks for doing all this, Max! When
you're ready, I can
> deploy
> >> to
> >>>>>>> some
> >>>>>>>>>>> of
> >>>>>>>>>>>> our environments to verify. Just let
me know.
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Thu, Jun 8, 2017 at 8:42 AM, Maxime
Beauchemin <
> >>>>>>>>>>>> maximebeauchemin@gmail.com <mailto:maximebeauchemin@gmail.com
> >>
> >>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> I branched off `v1-8-test` so all
should be good. I just
> didn't
> >>>> know
> >>>>>>>>>>> if I
> >>>>>>>>>>>>> could move ahead with that branch
just yet so I branched
> off. I
> >>>> just
> >>>>>>>>>>> got
> >>>>>>>>>>>>> back on `v1-8-test` and pushed what
I have to Apache
> [somehow I
> >>>> had
> >>>>>>>>>> to
> >>>>>>>>>>>>> rebase meaning someone added something
over the past 24
> hours].
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I just set `Fix Version` of AIRFLOW-1294
to 1.8.2 and set it
> to
> >>>>>>>>>>> blocker.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I cherry-picked all the commits
targeting 1.8.2 I could
> without
> >>>>>>>>>> getting
> >>>>>>>>>>>>> into conflicts. I'd only work at
resolving conflicts on
> >>>> blockers. My
> >>>>>>>>>>> plan
> >>>>>>>>>>>>> is to add only the 2 rows in red
for RC1.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Here's the current output of `airflow-jira
1.8.2`:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> ISSUE ID          |TYPE        ||PRIORITY
 ||STATUS
> >>>> |DESCRIPTION
> >>>>>>>>>>>>>                                |MERGED|PR
> >>>> |COMMITAIRFLOW-1294
> >>>>>>>>>>>>> |Bug         ||Blocker   ||Open
     |Backfills can loose
> tasks
> >>>> to
> >>>>>>>>>>>>> execute due to tasks |0     |- 
   |-
> >>>>>>>>>>>>> AIRFLOW-1291      |Bug         ||Blocker
    ||Open
> >> |Update
> >>>>>>>>>>>>> NOTICE and LICENSE files to meet
ASF specs |0     |-     |-
> >>>>>>>>>>>>> AIRFLOW-1290      |Bug         ||Major
    ||Resolved
> |Change
> >>>> docs
> >>>>>>>>>>>>> author from "Maxime Beauchemin"
to "Ap|1     |#na
> >>>>>>>>>>>>> |fc33c040605e7b0121739af841790336bdfa3948
> >>>>>>>>>>>>> AIRFLOW-1282      |Bug         ||Major
    ||Resolved  |Fix
> >> known
> >>>>>>>>>>>>> event column sorting           
        |1     |#2350
> >>>>>>>>>>>>> |2e9eee3697f6261de61c77ba584ace14b22751f0
> >>>>>>>>>>>>> AIRFLOW-1281      |Bug         ||Minor
    ||Resolved  |Sort
> >>>>>>>>>> variables
> >>>>>>>>>>>>> by key field by default        
   |1     |#2347
> >>>>>>>>>>>>> |57d5bcdad4202fd8a94be07b4aeacee065801481
> >>>>>>>>>>>>> AIRFLOW-1275      |Bug         ||Major
    ||In Progress|Fix
> >>>>>>> `airflow
> >>>>>>>>>>>>> pool` command exception        
     |0     |-     |-
> >>>>>>>>>>>>> AIRFLOW-1274      |Bug         ||Major
    ||Open
> >>>> |HttpSensor
> >>>>>>>>>>>>> parameter params is overriding BaseOper|0
    |-     |-
> >>>>>>>>>>>>> AIRFLOW-1266      |Bug         ||Major
    ||Open      |Long
> >> task
> >>>>>>>>>>>>> names are truncated in gannt view
      |1     |#2345
> >>>>>>>>>>>>> |7f33f6e2fe4149e547860fdb237da26a5b7023ec
> >>>>>>>>>>>>> AIRFLOW-1263      |Bug         ||Major
    ||Resolved
> |Airflow
> >>>>>>>>>> Charts
> >>>>>>>>>>>>> Pages Should Have Dynamic Heights
 |1     |#2344
> >>>>>>>>>>>>> |acd0166d8c7f5639898791848d985f6441b3b5b5
> >>>>>>>>>>>>> AIRFLOW-1244      |Bug         ||Major
    ||Resolved
> |Forbid
> >>>>>>>>>>>>> creation of a pool with empty name
        |1     |#2324
> >>>>>>>>>>>>> |802fc1549d0f173c646794f2476156d34383adc0
> >>>>>>>>>>>>> AIRFLOW-1243      |Bug         ||Minor
    ||Resolved  |DAGs
> >>>> table
> >>>>>>>>>> has
> >>>>>>>>>>>>> no default entries to show     
   |1     |#2323
> >>>>>>>>>>>>> |1232b6a4fd691e0faf081d4cf792a6d96d33b595
> >>>>>>>>>>>>> AIRFLOW-1227      |Bug         ||Minor
    ||Resolved
> |Remove
> >>>> empty
> >>>>>>>>>>>>> column on the Logs view        
     |1     |#2310
> >>>>>>>>>>>>> |b0ba3c91044b2abd92289d9b3f7d913060ef6c31
> >>>>>>>>>>>>> AIRFLOW-1226      |Bug         ||Minor
    ||Resolved
> |Remove
> >>>> empty
> >>>>>>>>>>>>> column on the Jobs view        
     |1     |#2309
> >>>>>>>>>>>>> |c406652bd4c8f8099841c69f2d1ccc9678d77bfd
> >>>>>>>>>>>>> AIRFLOW-1221      |Bug         ||Major
    ||Resolved  |Fix
> >>>>>>>>>>>>> DatabricksSubmitRunOperator Templating
       |0     |-
>  |-
> >>>>>>>>>>>>> AIRFLOW-1217      |Bug         ||Major
    ||Resolved
> |Enable
> >>>>>>>>>> logging
> >>>>>>>>>>>>> in Sqoop hook                  
   |0     |-     |-
> >>>>>>>>>>>>> AIRFLOW-1213      |Bug         ||Major
    ||Resolved  |Add
> >>>> hcatalog
> >>>>>>>>>>>>> parameters to the sqoop operator/hook|0
    |-     |-
> >>>>>>>>>>>>> AIRFLOW-1203      |Bug         ||Major
    ||Closed    |Tests
> >>>>>>> failing
> >>>>>>>>>>>>> after oauth upgrade            
    |0     |-     |-
> >>>>>>>>>>>>> AIRFLOW-1201      |Bug         ||Major
    ||Resolved
> |Update
> >>>>>>>>>>>>> deprecated 'nose-parameterized'
library to |0     |-     |-
> >>>>>>>>>>>>> AIRFLOW-1200      |Bug         ||Major
    ||Resolved
> |Forbid
> >>>>>>>>>>>>> creation of a variable with an empty
key   |1     |#2299
> >>>>>>>>>>>>> |cac7d4cb571a4c1f7ba53360765ea6c640c719df
> >>>>>>>>>>>>> AIRFLOW-1199      |Bug         ||Major
    ||Resolved
> |Create
> >>>> modal
> >>>>>>>>>>>>> looks like edit modal (edit tab
selec|1     |#2293
> >>>>>>>>>>>>> |51a83cce96d41eb1b9df9f82188c36c7e12061df
> >>>>>>>>>>>>> AIRFLOW-1197      |Bug         ||Major
    ||Open
> |Contrib
> >>>>>>> Spark
> >>>>>>>>>>>>> Submit Hook on_kill error      
    |0     |-     |-
> >>>>>>>>>>>>> AIRFLOW-1192      |New Feature ||Major
    ||Resolved  |Add
> >>>> support
> >>>>>>>>>> to
> >>>>>>>>>>>>> run Zeppelin Notebooks with Qubole
|1     |#2322
> >>>>>>>>>>>>> |bf966e63be2deb19fa2824efdfe854736318f1b5
> >>>>>>>>>>>>> AIRFLOW-1191      |Improvement ||Major
    ||Open
> |Contrib
> >>>>>>> Spark
> >>>>>>>>>>>>> Submit hook should permit override
o|0     |-     |-
> >>>>>>>>>>>>> AIRFLOW-1186      |Bug         ||Major
    ||Resolved  |Task
> >>>>>>> duration
> >>>>>>>>>>>>> shows weird graph for some dags
    |1     |#2284
> >>>>>>>>>>>>> |5f3ee52da7ca4f1fca8a84b3281b31aa6241514e
> >>>>>>>>>>>>> AIRFLOW-1180      |Bug         ||Major
    ||Resolved  |Fix
> >>>>>>> flask-wtf
> >>>>>>>>>>>>> version in setup.py for test_csrf_re|1
    |#2280
> >>>>>>>>>>>>> |93b8e9643cc273ec0ecc063b9f851d9a8012cd03
> >>>>>>>>>>>>> AIRFLOW-1179      |Bug         ||Major
    ||Closed
> |Pandas
> >>>> 0.20
> >>>>>>>>>>>>> broke Google BigQuery hook     
      |1     |#2279
> >>>>>>>>>>>>> |c7c0b72d934fec10d890f7a3bd8fd06c0e2f44bd
> >>>>>>>>>>>>> AIRFLOW-1170      |Bug         ||Major
    ||Resolved
> >> |DbApiHook
> >>>>>>>>>>>>> insert rows does not use prepared
statem|1     |#2270
> >>>>>>>>>>>>> |bb56805e3801c7f8d49acd8095fd3099609d6fe5
> >>>>>>>>>>>>> AIRFLOW-1168      |Bug         ||Major
    ||Resolved
> >> |DbApiHook
> >>>>>>>>>> does
> >>>>>>>>>>>>> not close connections when an excep|1
    |#2269
> >>>>>>>>>>>>> |777f181f81fa6a383a79b71fdcb9c4f4e77c785a
> >>>>>>>>>>>>> AIRFLOW-1167      |Bug         ||Major
    ||Resolved
> |FTPHook
> >>>> file
> >>>>>>>>>>>>> modification date does not support
op|1     |#2268
> >>>>>>>>>>>>> |272c2f52142c167685646169431eb9dab85b7810
> >>>>>>>>>>>>> AIRFLOW-1166      |Bug         ||Major
    ||Resolved
> >>>>>>>>>>>>> |_change_state_for_tis_without_dagrun
can take very|1
> >> |#2267
> >>>>>>>>>>>>> |2389a8a7e89ea949e8f9674a3d3b18d2142dfece
> >>>>>>>>>>>>> AIRFLOW-1160      |Bug         ||Major
    ||Resolved
> |Upadte
> >>>>>>>>>>>>> SparkSubmitOperator parameters 
           |1     |#2265
> >>>>>>>>>>>>> |69bd26913824c5c796a0a828a5c43ab39d0902f8
> >>>>>>>>>>>>> AIRFLOW-1150      |Bug         ||Major
    ||Resolved  |Fix
> >>>> scripts
> >>>>>>>>>>>>> execution in SparkSql hook     
      |1     |#2259
> >>>>>>>>>>>>> |093b2f04f7a010107fe6dfbc0658a53a1b8467e6
> >>>>>>>>>>>>> AIRFLOW-1149      |Bug         ||Major
    ||Resolved  |Allow
> >>>> custom
> >>>>>>>>>>>>> filters to be added to jinja2  
     |1     |#2258
> >>>>>>>>>>>>> |9692510c4e067d0e81fd8d5893bfb4a0e53941e2
> >>>>>>>>>>>>> AIRFLOW-1145      |Bug         ||Minor
    ||Resolved
> >>>>>>>>>>>>> |Closest_date_partition not working
with before = T|1
> >> |#2257
> >>>>>>>>>>>>> |f446c0883cab30482436c47b867c4a440412f1ff
> >>>>>>>>>>>>> AIRFLOW-1136      |Bug         ||Minor
    ||Resolved
> |Invalid
> >>>>>>>>>>>>> parameters are not captured for
Sqoop oper|0     |-     |-
> >>>>>>>>>>>>> AIRFLOW-1119      |Bug         ||Major
    ||Resolved
> >> |Redshift
> >>>> to
> >>>>>>>>>> S3
> >>>>>>>>>>>>> operator - headers not on first
row|1     |#2245
> >>>>>>>>>>>>> |6de5330d5336fb2455eea75c78e95f126320a5b0
> >>>>>>>>>>>>> AIRFLOW-1089      |Improvement ||Major
    ||Resolved  |Add
> >> Spark
> >>>>>>>>>>>>> application arguments to SparkSubmitOper|1
    |#2229
> >>>>>>>>>>>>> |b4e9eb8310957cac47f6964662ce16496061db2c
> >>>>>>>>>>>>> AIRFLOW-1081      |Bug         ||Major
    ||Resolved  |Task
> >>>>>>> duration
> >>>>>>>>>>>>> page is slow                   
    |0     |-     |-
> >>>>>>>>>>>>> AIRFLOW-1078      |Bug         ||Major
    ||Resolved
> >>>> |Latest_runs
> >>>>>>>>>>>>> endpoint broken in old flask versions
|1     |#2224
> >>>>>>>>>>>>> |a4083f367b1f0f6525bcd011122d854626ea6768
> >>>>>>>>>>>>> AIRFLOW-1074      |Bug         ||Major
    ||Open      |Do
> not
> >>>> count
> >>>>>>>>>>>>> queued tasks in scheduler concurrency|1
    |#2221
> >>>>>>>>>>>>> |7a0284149a1dde4dc1d65a1d016eadf619e5320e
> >>>>>>>>>>>>> AIRFLOW-1064      |Improvement ||Major
    ||Resolved
> >>>>>>>>>>>>> |TaskInstanceModelView is slow 
                   |1
> >> |#2215
> >>>>>>>>>>>>> |a2c18a58d0ba14101ddb70403fb95feda73cf21a
> >>>>>>>>>>>>> AIRFLOW-1047      |Bug         ||Major
    ||Resolved
> |Airflow
> >>>> logs
> >>>>>>>>>>>>> vulnerable to XSS              
     |0     |-     |-
> >>>>>>>>>>>>> AIRFLOW-1041      |Bug         ||Major
    ||Resolved
> >>>>>>>>>> |DockerOperator
> >>>>>>>>>>>>> replaces its xcom_push method with
|0     |-     |-
> >>>>>>>>>>>>> AIRFLOW-1038      |Bug         ||Major
    ||Resolved
> |Specify
> >>>>>>>>>> celery
> >>>>>>>>>>>>> serializers explicitly and pin vers|1
    |#na
> >>>>>>>>>>>>> |d1c64ab7b4ee6215900a15849e890ed1b408f5f5
> >>>>>>>>>>>>> AIRFLOW-1036      |Improvement ||Major
    ||Resolved
> >>>> |Exponential
> >>>>>>>>>>>>> backoff should use randomization
     |1     |#2262
> >>>>>>>>>>>>> |b4ee88aa6d71145daf91fc71763950ede16f8383
> >>>>>>>>>>>>> AIRFLOW-1007      |Bug         ||Major
    ||Resolved  |Jinja
> >>>>>>> sandbox
> >>>>>>>>>>>>> is vulnerable to RCE           
    |0     |-     |-
> >>>>>>>>>>>>> AIRFLOW-993       |Bug         ||Minor
    ||Resolved  |Dags
> >>>> should
> >>>>>>>>>>>>> modify the start date and end date
of |1     |#2157
> >>>>>>>>>>>>> |9fca409cebfd04ce8580c4657099a6db340eb7eb
> >>>>>>>>>>>>> AIRFLOW-945       |Bug         ||Major
    ||Resolved
> |Revert
> >>>>>>>>>>>>> psycopg2 workaround when psycopg2
2.7.1 is |0     |-     |-
> >>>>>>>>>>>>> AIRFLOW-935       |Bug         ||Major
    ||Resolved
> >>>> |Impossible
> >>>>>>> to
> >>>>>>>>>>>>> use plugin executors           
    |0     |-     |-
> >>>>>>>>>>>>> AIRFLOW-860       |Bug         ||Major
    ||Resolved
> >> |Circular
> >>>>>>>>>>>>> module dependency prevents loading
of cus|0     |-     |-
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Max
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Thu, Jun 8, 2017 at 12:18 AM,
Bolke de Bruin <
> >>>> bdbruin@gmail.com <mailto:bdbruin@gmail.com>>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hi Max,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Sounds good. Couple of things:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> * Can I suggest using the v1-8-test
branch as the branch to
> be
> >>>> used
> >>>>>>>>>>> for
> >>>>>>>>>>>>>> preparing the rc? If we hit
RC then move it over to
> >> v1-8-stable?
> >>>>>>>>>>>>> V1-8-test
> >>>>>>>>>>>>>> already had some fixes in that
should land in 1.8.2 and the
> RC
> >>>>>>>>>> should
> >>>>>>>>>>>> be
> >>>>>>>>>>>>>> tagged in the stable branch.
That also reduces to amount of
> >>>> merge
> >>>>>>>>>>>>> conflicts
> >>>>>>>>>>>>>> probably as many have been merged.
Where did you branch off
> >>>> from?
> >>>>>>>>>>>>> Anyways,
> >>>>>>>>>>>>>> see also the release management
thing on the wiki.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Blocker(!)
> >>>>>>>>>>>>>> * In the backfills we can loose
tasks to execute due to a
> task
> >>>>>>>>>>> setting
> >>>>>>>>>>>>> its
> >>>>>>>>>>>>>> own state to NONE if concurrency
limits are reached, this
> >> makes
> >>>>>>>>>> them
> >>>>>>>>>>>> fall
> >>>>>>>>>>>>>> outside of the scope the backfill
is managing hence they
> will
> >>>> not
> >>>>>>>>>> be
> >>>>>>>>>>>>>> executed (https://issues.apache.org/
> jira/browse/AIRFLOW-1294
> >> <
> >>>> https://issues.apache.org/jira/browse/AIRFLOW-1294> <
> >>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1294
<
> >>>> https://issues.apache.org/jira/browse/AIRFLOW-1294>>). Setting
> >>>>>>>>>> itself
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>> NONE should probably be “CONCURRENCY_REACHED”
(new state). I
> >>>> have
> >>>>>>>>>>>> marked
> >>>>>>>>>>>>> it
> >>>>>>>>>>>>>> as a blocker as we had multiple
people hitting the issue,
> but
> >> I
> >>>>>>>>>> need
> >>>>>>>>>>>> 1-2
> >>>>>>>>>>>>>> days to get a patch. Feel free
to downgrade to critical if
> you
> >>>> like
> >>>>>>>>>>> :).
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Cheers
> >>>>>>>>>>>>>> Bolke
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On 8 Jun 2017, at 02:35,
Maxime Beauchemin <
> >>>>>>>>>>>> maximebeauchemin@gmail.com <mailto:maximebeauchemin@gmail.com
> >
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> What a pleasant, mind numbing
afternoon doing some release
> >>>>>>>>>>> management
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Notes:
> >>>>>>>>>>>>>>> * Added a warning that the
package name has changed on Pypi
> >>>>>>>>>>>>>>> <https://pypi.python.org/pypi/airflow
<
> >>>> https://pypi.python.org/pypi/airflow>>
> >>>>>>>>>>>>>>> * Removed references to
my name here
> >>>>>>>>>>>>>>> <https://github.com/apache/incubator-airflow/pull/2352
<
> >>>> https://github.com/apache/incubator-airflow/pull/2352>> and
> >>>>>>>>>> merged
> >>>>>>>>>>>>>>> * Addressed John D. Ament's
concerns here
> >>>>>>>>>>>>>>> <https://github.com/apache/incubator-airflow/pull/2354
<
> >>>> https://github.com/apache/incubator-airflow/pull/2354>>, please
> >>>>>>>>>>>>> review!
> >>>>>>>>>>>>>>> * "footable" appears to
have been removed, not a problem
> >>>> anymore
> >>>>>>>>>>>>>>> * that `airflow-jira is
a god send! thanks Bolke.
> >>>>>>>>>>>>>>> * reviewed list of Airbnb's
production cherries and flagged
> >>>> those
> >>>>>>>>>>> as
> >>>>>>>>>>>>> `Fix
> >>>>>>>>>>>>>>> Version == 1.8.2`
> >>>>>>>>>>>>>>> * Started branch v1-8-2.rc1
and started picking cherries
> >> using
> >>>>>>>>>>>>>>> `airflow-jira compare 1.8.2`
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I'll finish going through
picking everything that targeted
> >>>> 1.8.2
> >>>>>>>>>>> that
> >>>>>>>>>>>>>> does
> >>>>>>>>>>>>>>> not create merge conflict.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> If there's anything flagged
as "blocker" that generates
> merge
> >>>>>>>>>>>> conflict,
> >>>>>>>>>>>>>>> I'll go case by case about
it.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Soon after, I should be
able to announce 1.8.2 RC1,
> hopefully
> >>>>>>>>>>>> sometime
> >>>>>>>>>>>>>>> tomorrow or Friday.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Let me know if there's anything
else I'm missing that I
> >> should
> >>>>>>>>>>>>> consider.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Cheers!
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Max
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>
> >>>>
> >>>>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message