airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bolke de Bruin <bdbr...@gmail.com>
Subject Re: Release Manager for 1.8.2?
Date Fri, 23 Jun 2017 18:33:57 GMT
Yes.

Bolke.

> On 23 Jun 2017, at 19:55, Maxime Beauchemin <maximebeauchemin@gmail.com> wrote:
> 
> I appended my public key to the KEYS file.
> 
> I'm not sure if I understand your 3rd point, should I remove all the
> previous tarballs and related files from https://dist.apache.org/repos/
> dist/dev/incubator/airflow/ ?
> 
> Max
> 
> On Fri, Jun 23, 2017 at 5:48 AM, Bolke de Bruin <bdbruin@gmail.com> wrote:
> 
>> Hi Max,
>> 
>> Looking on a phone, so I might not have spotted everything.
>> 
>> - Tag looks good
>> - Your public GPG key has not been added yet to
>> https://dist.apache.org/repos/dist/release/incubator/airflow/KEYS <
>> https://dist.apache.org/repos/dist/release/incubator/airflow/KEYS>
>> - Please remove the old tar balls from https://dist.apache.org/repos/
>> dist/dev/incubator/airflow/ <https://dist.apache.org/
>> repos/dist/dev/incubator/airflow/> is causes confusion for some reason at
>> the IPMC
>> 
>> @chris please take a look as well.
>> 
>> Bolke
>> 
>>> On 23 Jun 2017, at 00:34, Maxime Beauchemin <maximebeauchemin@gmail.com>
>> wrote:
>>> 
>>> Allllllllright, so I cherry-picked that last one, some of the other last
>>> minute ones had conflicts, and I bailed on all conflictual non-blocking
>>> issues.
>>> 
>>> So here is the new tag, it's on the v1-8-test branch:
>>> https://github.com/apache/incubator-airflow/releases/tag/1.8.2rc2
>>> 
>>> The tarball, hashes and signature are here:
>>> https://dist.apache.org/repos/dist/dev/incubator/airflow/
>>> 
>>> Changelog:
>>> https://github.com/apache/incubator-airflow/blob/1.8.2rc2/CHANGELOG.txt
>>> 
>>> Can someone confirm this looks alright before I send an announcement
>> about
>>> 1.8.2rc2 to the list?
>>> 
>>> Max
>>> 
>>> On Wed, Jun 21, 2017 at 10:44 PM, Bolke de Bruin <bdbruin@gmail.com>
>> wrote:
>>> 
>>>> Please also include https://github.com/apache/
>> incubator-airflow/pull/2022
>>>> 
>>>> I have had reports it was also required for MySQL.
>>>> 
>>>> Bolke
>>>> 
>>>> Sent from my iPhone
>>>> 
>>>>> On 22 Jun 2017, at 00:03, Maxime Beauchemin <
>> maximebeauchemin@gmail.com>
>>>> wrote:
>>>>> 
>>>>> Ok I'll try to cherry-pick all of these, though if there are any merge
>>>>> conflicts I'll pass and move their Fix-Version to 1.9.0.
>>>>> 
>>>>> Max
>>>>> 
>>>>>> On Wed, Jun 21, 2017 at 1:20 AM, Bolke de Bruin <bdbruin@gmail.com>
>>>> wrote:
>>>>>> 
>>>>>> Hi max,
>>>>>> 
>>>>>> PR 2365 is merged into master and also into v1-8-test.
>>>>>> 
>>>>>> I have added:
>>>>>> 
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-935 <
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-935> and
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-860 <
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-860>
>>>>>> 
>>>>>> To the list fro 1.8.2. The PR for this has been in master for some
>> time:
>>>>>> https://github.com/apache/incubator-airflow/pull/2120 <
>>>>>> https://github.com/apache/incubator-airflow/pull/2120>
>>>>>> 
>>>>>> Can you consider it for 1.8.2? Basically I promised to push to the
>>>> matter
>>>>>> :-).
>>>>>> 
>>>>>> No blockers anymore as far as I know for 1.8.2.
>>>>>> 
>>>>>> Bolke
>>>>>> 
>>>>>> P.S. A friendly request if any of the committers do +1 or a LGTM
to a
>>>> PR,
>>>>>> please also merge the PR instead of leaving it to others. “airflow-pr
>>>> merge
>>>>>> XXX” is there to help.
>>>>>> 
>>>>>>> On 16 Jun 2017, at 15:11, Bolke de Bruin <bdbruin@gmail.com>
wrote:
>>>>>>> 
>>>>>>> It is now pinned. I think Cloudera messed up their release for
some
>>>>>> reason.
>>>>>>> 
>>>>>>> Builds are succeeding again.
>>>>>>> 
>>>>>>> Please note I have one blocker out for 1.8.2:
>>>> https://github.com/apache/
>>>>>> incubator-airflow/pull/2365 <https://github.com/apache/
>>>>>> incubator-airflow/pull/2365>
>>>>>>> 
>>>>>>> Bolke
>>>>>>> 
>>>>>>>> On 15 Jun 2017, at 16:43, Maxime Beauchemin <
>>>> maximebeauchemin@gmail.com
>>>>>> <mailto:maximebeauchemin@gmail.com>> wrote:
>>>>>>>> 
>>>>>>>> Awesome, thanks for looking into it. Is there a way we can
pin the
>>>>>> Cloudera
>>>>>>>> tarball to avoid surprises like this?
>>>>>>>> 
>>>>>>>> On Thu, Jun 15, 2017 at 7:17 AM, Bolke de Bruin <bdbruin@gmail.com
>>>>>> <mailto:bdbruin@gmail.com>> wrote:
>>>>>>>> 
>>>>>>>>> Found the issue for the permission denied part: beeline
is not
>>>>>> executable
>>>>>>>>> in the latest tarball from Cloudera. I’ll have a workaround
for
>> this
>>>>>> in a
>>>>>>>>> few minutes and will push it to Apache right away.
>>>>>>>>> 
>>>>>>>>> Bolke.
>>>>>>>>> 
>>>>>>>>>> On 14 Jun 2017, at 18:15, Bolke de Bruin <bdbruin@gmail.com
>>>> <mailto:
>>>>>> bdbruin@gmail.com>> wrote:
>>>>>>>>>> 
>>>>>>>>>> Hi Max,
>>>>>>>>>> 
>>>>>>>>>> I have created https://github.com/apache/
>>>> incubator-airflow/pull/2365
>>>>>> <https://github.com/apache/incubator-airflow/pull/2365> <
>>>>>>>>> https://github.com/apache/incubator-airflow/pull/2365
<
>>>>>> https://github.com/apache/incubator-airflow/pull/2365>> for
>>>> AIRFLOW-1296,
>>>>>>>>> which I think should be treated as a blocker.
>>>>>>>>>> 
>>>>>>>>>> I think Travis’ is failing due to a dependency
upgrade of Dask.
>>>>>>>>>> 
>>>>>>>>>> The permission denied error seems to come from a
new Travis config
>>>>>> that
>>>>>>>>> does not allow the creation of “/user/hive/warehouse”
by the normal
>>>>>> user.
>>>>>>>>> Probably a “sudo mkdir “ and a “sudo chown” will
help here. Let me
>>>>>> check.
>>>>>>>>>> 
>>>>>>>>>> Bolke
>>>>>>>>>> 
>>>>>>>>>>> On 14 Jun 2017, at 00:26, Maxime Beauchemin <
>>>>>> maximebeauchemin@gmail.com <mailto:maximebeauchemin@gmail.com>
>>>>>>>>> <mailto:maximebeauchemin@gmail.com <mailto:maximebeauchemin@
>>>> gmail.com>>>
>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> A quick update on my progress.
>>>>>>>>>>> 
>>>>>>>>>>> I ran through the "Releasing Airflow" wiki playbook
only to
>>>> realize I
>>>>>>>>> had
>>>>>>>>>>> published a version that failed the Travis build
>>>>>>>>>>> <https://travis-ci.org/mistercrunch/incubator-
>>>> airflow/jobs/242532965
>>>>>> <https://travis-ci.org/mistercrunch/incubator-airflow/jobs/242532965>
>> <
>>>>>>>>> https://travis-ci.org/mistercrunch/incubator-
>> airflow/jobs/242532965
>>>> <
>>>>>> https://travis-ci.org/mistercrunch/incubator-airflow/jobs/242532965
>>>>> 
>>>>>>>>>>> afterwards. I've
>>>>>>>>>>> been updating the wiki as I go and it seems like
it will be
>> better
>>>>>> the
>>>>>>>>> next
>>>>>>>>>>> time around. I'm getting "Permission denied"
on the Hive hook
>>>>>>>>> subprocess if
>>>>>>>>>>> anyone can shed some light on that.
>>>>>>>>>>> 
>>>>>>>>>>> So I need to fix the unit tests at this point,
and probably go
>>>>>> straight
>>>>>>>>> to
>>>>>>>>>>> `rc2` to avoid changing the files out there for
`rc1`.
>>>>>>>>>>> 
>>>>>>>>>>> Max
>>>>>>>>>>> 
>>>>>>>>>>> On Thu, Jun 8, 2017 at 4:54 PM, Ruslan Dautkhanov
<
>>>>>> dautkhanov@gmail.com <mailto:dautkhanov@gmail.com>
>>>>>>>>> <mailto:dautkhanov@gmail.com <mailto:dautkhanov@gmail.com>>>
>>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> It would be great if somebody would have
a look at following 3
>>>> jiras
>>>>>>>>>>>> 
>>>>>>>>>>>> I've flagged
>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013
<
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013> <
>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013 <
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013>>
>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1178
<
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1178>
>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1055
>>>>>>>>>>>> 
>>>>>>>>>>>> Two of them were Blockers for one of previous
1.8 releases but
>>>> were
>>>>>>>>>>>> de-escalated because no resources to fix
them.
>>>>>>>>>>>> 
>>>>>>>>>>>> I'm currently Assignee on AIRFLOW-1055 -
can't unassign myself -
>>>>>> feel
>>>>>>>>> free
>>>>>>>>>>>> to scratch my name there.
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> --
>>>>>>>>>>>> Ruslan Dautkhanov
>>>>>>>>>>>> 
>>>>>>>>>>>> On Fri, Jun 9, 2017 at 12:23 AM, Maxime Beauchemin
<
>>>>>>>>>>>> maximebeauchemin@gmail.com <mailto:maximebeauchemin@gmail.com>>
>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> Cool. I'll let everyone know when RC1
is out.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> @community, if you think anything else
should be a blocker for
>>>>>> 1.8.2
>>>>>>>>>>>> please
>>>>>>>>>>>>> flag it in Jira very soon or it will
miss the release train!
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Max
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Thu, Jun 8, 2017 at 9:06 AM, Chris
Riccomini <
>>>>>>>>> criccomini@apache.org <mailto:criccomini@apache.org>>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Thanks for doing all this, Max! When
you're ready, I can
>> deploy
>>>> to
>>>>>>>>> some
>>>>>>>>>>>>> of
>>>>>>>>>>>>>> our environments to verify. Just
let me know.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Thu, Jun 8, 2017 at 8:42 AM, Maxime
Beauchemin <
>>>>>>>>>>>>>> maximebeauchemin@gmail.com <mailto:maximebeauchemin@gmail.com
>>>> 
>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> I branched off `v1-8-test` so
all should be good. I just
>> didn't
>>>>>> know
>>>>>>>>>>>>> if I
>>>>>>>>>>>>>>> could move ahead with that branch
just yet so I branched
>> off. I
>>>>>> just
>>>>>>>>>>>>> got
>>>>>>>>>>>>>>> back on `v1-8-test` and pushed
what I have to Apache
>> [somehow I
>>>>>> had
>>>>>>>>>>>> to
>>>>>>>>>>>>>>> rebase meaning someone added
something over the past 24
>> hours].
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> I just set `Fix Version` of AIRFLOW-1294
to 1.8.2 and set it
>> to
>>>>>>>>>>>>> blocker.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> I cherry-picked all the commits
targeting 1.8.2 I could
>> without
>>>>>>>>>>>> getting
>>>>>>>>>>>>>>> into conflicts. I'd only work
at resolving conflicts on
>>>>>> blockers. My
>>>>>>>>>>>>> plan
>>>>>>>>>>>>>>> is to add only the 2 rows in
red for RC1.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Here's the current output of
`airflow-jira 1.8.2`:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> ISSUE ID          |TYPE     
  ||PRIORITY  ||STATUS
>>>>>> |DESCRIPTION
>>>>>>>>>>>>>>>                             
 |MERGED|PR
>>>>>> |COMMITAIRFLOW-1294
>>>>>>>>>>>>>>> |Bug         ||Blocker   ||Open
     |Backfills can loose
>> tasks
>>>>>> to
>>>>>>>>>>>>>>> execute due to tasks |0     |-
    |-
>>>>>>>>>>>>>>> AIRFLOW-1291      |Bug      
  ||Blocker     ||Open
>>>> |Update
>>>>>>>>>>>>>>> NOTICE and LICENSE files to meet
ASF specs |0     |-     |-
>>>>>>>>>>>>>>> AIRFLOW-1290      |Bug      
  ||Major     ||Resolved
>> |Change
>>>>>> docs
>>>>>>>>>>>>>>> author from "Maxime Beauchemin"
to "Ap|1     |#na
>>>>>>>>>>>>>>> |fc33c040605e7b0121739af841790336bdfa3948
>>>>>>>>>>>>>>> AIRFLOW-1282      |Bug      
  ||Major     ||Resolved  |Fix
>>>> known
>>>>>>>>>>>>>>> event column sorting        
           |1     |#2350
>>>>>>>>>>>>>>> |2e9eee3697f6261de61c77ba584ace14b22751f0
>>>>>>>>>>>>>>> AIRFLOW-1281      |Bug      
  ||Minor     ||Resolved  |Sort
>>>>>>>>>>>> variables
>>>>>>>>>>>>>>> by key field by default     
      |1     |#2347
>>>>>>>>>>>>>>> |57d5bcdad4202fd8a94be07b4aeacee065801481
>>>>>>>>>>>>>>> AIRFLOW-1275      |Bug      
  ||Major     ||In Progress|Fix
>>>>>>>>> `airflow
>>>>>>>>>>>>>>> pool` command exception     
        |0     |-     |-
>>>>>>>>>>>>>>> AIRFLOW-1274      |Bug      
  ||Major     ||Open
>>>>>> |HttpSensor
>>>>>>>>>>>>>>> parameter params is overriding
BaseOper|0     |-     |-
>>>>>>>>>>>>>>> AIRFLOW-1266      |Bug      
  ||Major     ||Open      |Long
>>>> task
>>>>>>>>>>>>>>> names are truncated in gannt
view       |1     |#2345
>>>>>>>>>>>>>>> |7f33f6e2fe4149e547860fdb237da26a5b7023ec
>>>>>>>>>>>>>>> AIRFLOW-1263      |Bug      
  ||Major     ||Resolved
>> |Airflow
>>>>>>>>>>>> Charts
>>>>>>>>>>>>>>> Pages Should Have Dynamic Heights
 |1     |#2344
>>>>>>>>>>>>>>> |acd0166d8c7f5639898791848d985f6441b3b5b5
>>>>>>>>>>>>>>> AIRFLOW-1244      |Bug      
  ||Major     ||Resolved
>> |Forbid
>>>>>>>>>>>>>>> creation of a pool with empty
name         |1     |#2324
>>>>>>>>>>>>>>> |802fc1549d0f173c646794f2476156d34383adc0
>>>>>>>>>>>>>>> AIRFLOW-1243      |Bug      
  ||Minor     ||Resolved  |DAGs
>>>>>> table
>>>>>>>>>>>> has
>>>>>>>>>>>>>>> no default entries to show  
      |1     |#2323
>>>>>>>>>>>>>>> |1232b6a4fd691e0faf081d4cf792a6d96d33b595
>>>>>>>>>>>>>>> AIRFLOW-1227      |Bug      
  ||Minor     ||Resolved
>> |Remove
>>>>>> empty
>>>>>>>>>>>>>>> column on the Logs view     
        |1     |#2310
>>>>>>>>>>>>>>> |b0ba3c91044b2abd92289d9b3f7d913060ef6c31
>>>>>>>>>>>>>>> AIRFLOW-1226      |Bug      
  ||Minor     ||Resolved
>> |Remove
>>>>>> empty
>>>>>>>>>>>>>>> column on the Jobs view     
        |1     |#2309
>>>>>>>>>>>>>>> |c406652bd4c8f8099841c69f2d1ccc9678d77bfd
>>>>>>>>>>>>>>> AIRFLOW-1221      |Bug      
  ||Major     ||Resolved  |Fix
>>>>>>>>>>>>>>> DatabricksSubmitRunOperator Templating
       |0     |-
>> |-
>>>>>>>>>>>>>>> AIRFLOW-1217      |Bug      
  ||Major     ||Resolved
>> |Enable
>>>>>>>>>>>> logging
>>>>>>>>>>>>>>> in Sqoop hook               
      |0     |-     |-
>>>>>>>>>>>>>>> AIRFLOW-1213      |Bug      
  ||Major     ||Resolved  |Add
>>>>>> hcatalog
>>>>>>>>>>>>>>> parameters to the sqoop operator/hook|0
    |-     |-
>>>>>>>>>>>>>>> AIRFLOW-1203      |Bug      
  ||Major     ||Closed    |Tests
>>>>>>>>> failing
>>>>>>>>>>>>>>> after oauth upgrade         
       |0     |-     |-
>>>>>>>>>>>>>>> AIRFLOW-1201      |Bug      
  ||Major     ||Resolved
>> |Update
>>>>>>>>>>>>>>> deprecated 'nose-parameterized'
library to |0     |-     |-
>>>>>>>>>>>>>>> AIRFLOW-1200      |Bug      
  ||Major     ||Resolved
>> |Forbid
>>>>>>>>>>>>>>> creation of a variable with an
empty key   |1     |#2299
>>>>>>>>>>>>>>> |cac7d4cb571a4c1f7ba53360765ea6c640c719df
>>>>>>>>>>>>>>> AIRFLOW-1199      |Bug      
  ||Major     ||Resolved
>> |Create
>>>>>> modal
>>>>>>>>>>>>>>> looks like edit modal (edit tab
selec|1     |#2293
>>>>>>>>>>>>>>> |51a83cce96d41eb1b9df9f82188c36c7e12061df
>>>>>>>>>>>>>>> AIRFLOW-1197      |Bug      
  ||Major     ||Open
>> |Contrib
>>>>>>>>> Spark
>>>>>>>>>>>>>>> Submit Hook on_kill error   
       |0     |-     |-
>>>>>>>>>>>>>>> AIRFLOW-1192      |New Feature
||Major     ||Resolved  |Add
>>>>>> support
>>>>>>>>>>>> to
>>>>>>>>>>>>>>> run Zeppelin Notebooks with Qubole
|1     |#2322
>>>>>>>>>>>>>>> |bf966e63be2deb19fa2824efdfe854736318f1b5
>>>>>>>>>>>>>>> AIRFLOW-1191      |Improvement
||Major     ||Open
>> |Contrib
>>>>>>>>> Spark
>>>>>>>>>>>>>>> Submit hook should permit override
o|0     |-     |-
>>>>>>>>>>>>>>> AIRFLOW-1186      |Bug      
  ||Major     ||Resolved  |Task
>>>>>>>>> duration
>>>>>>>>>>>>>>> shows weird graph for some dags
    |1     |#2284
>>>>>>>>>>>>>>> |5f3ee52da7ca4f1fca8a84b3281b31aa6241514e
>>>>>>>>>>>>>>> AIRFLOW-1180      |Bug      
  ||Major     ||Resolved  |Fix
>>>>>>>>> flask-wtf
>>>>>>>>>>>>>>> version in setup.py for test_csrf_re|1
    |#2280
>>>>>>>>>>>>>>> |93b8e9643cc273ec0ecc063b9f851d9a8012cd03
>>>>>>>>>>>>>>> AIRFLOW-1179      |Bug      
  ||Major     ||Closed
>> |Pandas
>>>>>> 0.20
>>>>>>>>>>>>>>> broke Google BigQuery hook  
         |1     |#2279
>>>>>>>>>>>>>>> |c7c0b72d934fec10d890f7a3bd8fd06c0e2f44bd
>>>>>>>>>>>>>>> AIRFLOW-1170      |Bug      
  ||Major     ||Resolved
>>>> |DbApiHook
>>>>>>>>>>>>>>> insert rows does not use prepared
statem|1     |#2270
>>>>>>>>>>>>>>> |bb56805e3801c7f8d49acd8095fd3099609d6fe5
>>>>>>>>>>>>>>> AIRFLOW-1168      |Bug      
  ||Major     ||Resolved
>>>> |DbApiHook
>>>>>>>>>>>> does
>>>>>>>>>>>>>>> not close connections when an
excep|1     |#2269
>>>>>>>>>>>>>>> |777f181f81fa6a383a79b71fdcb9c4f4e77c785a
>>>>>>>>>>>>>>> AIRFLOW-1167      |Bug      
  ||Major     ||Resolved
>> |FTPHook
>>>>>> file
>>>>>>>>>>>>>>> modification date does not support
op|1     |#2268
>>>>>>>>>>>>>>> |272c2f52142c167685646169431eb9dab85b7810
>>>>>>>>>>>>>>> AIRFLOW-1166      |Bug      
  ||Major     ||Resolved
>>>>>>>>>>>>>>> |_change_state_for_tis_without_dagrun
can take very|1
>>>> |#2267
>>>>>>>>>>>>>>> |2389a8a7e89ea949e8f9674a3d3b18d2142dfece
>>>>>>>>>>>>>>> AIRFLOW-1160      |Bug      
  ||Major     ||Resolved
>> |Upadte
>>>>>>>>>>>>>>> SparkSubmitOperator parameters
            |1     |#2265
>>>>>>>>>>>>>>> |69bd26913824c5c796a0a828a5c43ab39d0902f8
>>>>>>>>>>>>>>> AIRFLOW-1150      |Bug      
  ||Major     ||Resolved  |Fix
>>>>>> scripts
>>>>>>>>>>>>>>> execution in SparkSql hook  
         |1     |#2259
>>>>>>>>>>>>>>> |093b2f04f7a010107fe6dfbc0658a53a1b8467e6
>>>>>>>>>>>>>>> AIRFLOW-1149      |Bug      
  ||Major     ||Resolved  |Allow
>>>>>> custom
>>>>>>>>>>>>>>> filters to be added to jinja2
       |1     |#2258
>>>>>>>>>>>>>>> |9692510c4e067d0e81fd8d5893bfb4a0e53941e2
>>>>>>>>>>>>>>> AIRFLOW-1145      |Bug      
  ||Minor     ||Resolved
>>>>>>>>>>>>>>> |Closest_date_partition not working
with before = T|1
>>>> |#2257
>>>>>>>>>>>>>>> |f446c0883cab30482436c47b867c4a440412f1ff
>>>>>>>>>>>>>>> AIRFLOW-1136      |Bug      
  ||Minor     ||Resolved
>> |Invalid
>>>>>>>>>>>>>>> parameters are not captured for
Sqoop oper|0     |-     |-
>>>>>>>>>>>>>>> AIRFLOW-1119      |Bug      
  ||Major     ||Resolved
>>>> |Redshift
>>>>>> to
>>>>>>>>>>>> S3
>>>>>>>>>>>>>>> operator - headers not on first
row|1     |#2245
>>>>>>>>>>>>>>> |6de5330d5336fb2455eea75c78e95f126320a5b0
>>>>>>>>>>>>>>> AIRFLOW-1089      |Improvement
||Major     ||Resolved  |Add
>>>> Spark
>>>>>>>>>>>>>>> application arguments to SparkSubmitOper|1
    |#2229
>>>>>>>>>>>>>>> |b4e9eb8310957cac47f6964662ce16496061db2c
>>>>>>>>>>>>>>> AIRFLOW-1081      |Bug      
  ||Major     ||Resolved  |Task
>>>>>>>>> duration
>>>>>>>>>>>>>>> page is slow                
       |0     |-     |-
>>>>>>>>>>>>>>> AIRFLOW-1078      |Bug      
  ||Major     ||Resolved
>>>>>> |Latest_runs
>>>>>>>>>>>>>>> endpoint broken in old flask
versions |1     |#2224
>>>>>>>>>>>>>>> |a4083f367b1f0f6525bcd011122d854626ea6768
>>>>>>>>>>>>>>> AIRFLOW-1074      |Bug      
  ||Major     ||Open      |Do
>> not
>>>>>> count
>>>>>>>>>>>>>>> queued tasks in scheduler concurrency|1
    |#2221
>>>>>>>>>>>>>>> |7a0284149a1dde4dc1d65a1d016eadf619e5320e
>>>>>>>>>>>>>>> AIRFLOW-1064      |Improvement
||Major     ||Resolved
>>>>>>>>>>>>>>> |TaskInstanceModelView is slow
                    |1
>>>> |#2215
>>>>>>>>>>>>>>> |a2c18a58d0ba14101ddb70403fb95feda73cf21a
>>>>>>>>>>>>>>> AIRFLOW-1047      |Bug      
  ||Major     ||Resolved
>> |Airflow
>>>>>> logs
>>>>>>>>>>>>>>> vulnerable to XSS           
        |0     |-     |-
>>>>>>>>>>>>>>> AIRFLOW-1041      |Bug      
  ||Major     ||Resolved
>>>>>>>>>>>> |DockerOperator
>>>>>>>>>>>>>>> replaces its xcom_push method
with |0     |-     |-
>>>>>>>>>>>>>>> AIRFLOW-1038      |Bug      
  ||Major     ||Resolved
>> |Specify
>>>>>>>>>>>> celery
>>>>>>>>>>>>>>> serializers explicitly and pin
vers|1     |#na
>>>>>>>>>>>>>>> |d1c64ab7b4ee6215900a15849e890ed1b408f5f5
>>>>>>>>>>>>>>> AIRFLOW-1036      |Improvement
||Major     ||Resolved
>>>>>> |Exponential
>>>>>>>>>>>>>>> backoff should use randomization
     |1     |#2262
>>>>>>>>>>>>>>> |b4ee88aa6d71145daf91fc71763950ede16f8383
>>>>>>>>>>>>>>> AIRFLOW-1007      |Bug      
  ||Major     ||Resolved  |Jinja
>>>>>>>>> sandbox
>>>>>>>>>>>>>>> is vulnerable to RCE        
       |0     |-     |-
>>>>>>>>>>>>>>> AIRFLOW-993       |Bug      
  ||Minor     ||Resolved  |Dags
>>>>>> should
>>>>>>>>>>>>>>> modify the start date and end
date of |1     |#2157
>>>>>>>>>>>>>>> |9fca409cebfd04ce8580c4657099a6db340eb7eb
>>>>>>>>>>>>>>> AIRFLOW-945       |Bug      
  ||Major     ||Resolved
>> |Revert
>>>>>>>>>>>>>>> psycopg2 workaround when psycopg2
2.7.1 is |0     |-     |-
>>>>>>>>>>>>>>> AIRFLOW-935       |Bug      
  ||Major     ||Resolved
>>>>>> |Impossible
>>>>>>>>> to
>>>>>>>>>>>>>>> use plugin executors        
       |0     |-     |-
>>>>>>>>>>>>>>> AIRFLOW-860       |Bug      
  ||Major     ||Resolved
>>>> |Circular
>>>>>>>>>>>>>>> module dependency prevents loading
of cus|0     |-     |-
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Max
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Thu, Jun 8, 2017 at 12:18
AM, Bolke de Bruin <
>>>>>> bdbruin@gmail.com <mailto:bdbruin@gmail.com>>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Hi Max,
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Sounds good. Couple of things:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> * Can I suggest using the
v1-8-test branch as the branch to
>> be
>>>>>> used
>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>> preparing the rc? If we hit
RC then move it over to
>>>> v1-8-stable?
>>>>>>>>>>>>>>> V1-8-test
>>>>>>>>>>>>>>>> already had some fixes in
that should land in 1.8.2 and the
>> RC
>>>>>>>>>>>> should
>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>> tagged in the stable branch.
That also reduces to amount of
>>>>>> merge
>>>>>>>>>>>>>>> conflicts
>>>>>>>>>>>>>>>> probably as many have been
merged. Where did you branch off
>>>>>> from?
>>>>>>>>>>>>>>> Anyways,
>>>>>>>>>>>>>>>> see also the release management
thing on the wiki.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Blocker(!)
>>>>>>>>>>>>>>>> * In the backfills we can
loose tasks to execute due to a
>> task
>>>>>>>>>>>>> setting
>>>>>>>>>>>>>>> its
>>>>>>>>>>>>>>>> own state to NONE if concurrency
limits are reached, this
>>>> makes
>>>>>>>>>>>> them
>>>>>>>>>>>>>> fall
>>>>>>>>>>>>>>>> outside of the scope the
backfill is managing hence they
>> will
>>>>>> not
>>>>>>>>>>>> be
>>>>>>>>>>>>>>>> executed (https://issues.apache.org/
>> jira/browse/AIRFLOW-1294
>>>> <
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1294> <
>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1294
<
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1294>>). Setting
>>>>>>>>>>>> itself
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> NONE should probably be “CONCURRENCY_REACHED”
(new state). I
>>>>>> have
>>>>>>>>>>>>>> marked
>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>> as a blocker as we had multiple
people hitting the issue,
>> but
>>>> I
>>>>>>>>>>>> need
>>>>>>>>>>>>>> 1-2
>>>>>>>>>>>>>>>> days to get a patch. Feel
free to downgrade to critical if
>> you
>>>>>> like
>>>>>>>>>>>>> :).
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>>>>> Bolke
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On 8 Jun 2017, at 02:35,
Maxime Beauchemin <
>>>>>>>>>>>>>> maximebeauchemin@gmail.com <mailto:maximebeauchemin@gmail.com
>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> What a pleasant, mind
numbing afternoon doing some release
>>>>>>>>>>>>> management
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Notes:
>>>>>>>>>>>>>>>>> * Added a warning that
the package name has changed on Pypi
>>>>>>>>>>>>>>>>> <https://pypi.python.org/pypi/airflow
<
>>>>>> https://pypi.python.org/pypi/airflow>>
>>>>>>>>>>>>>>>>> * Removed references
to my name here
>>>>>>>>>>>>>>>>> <https://github.com/apache/incubator-airflow/pull/2352
<
>>>>>> https://github.com/apache/incubator-airflow/pull/2352>> and
>>>>>>>>>>>> merged
>>>>>>>>>>>>>>>>> * Addressed John D. Ament's
concerns here
>>>>>>>>>>>>>>>>> <https://github.com/apache/incubator-airflow/pull/2354
<
>>>>>> https://github.com/apache/incubator-airflow/pull/2354>>, please
>>>>>>>>>>>>>>> review!
>>>>>>>>>>>>>>>>> * "footable" appears
to have been removed, not a problem
>>>>>> anymore
>>>>>>>>>>>>>>>>> * that `airflow-jira
is a god send! thanks Bolke.
>>>>>>>>>>>>>>>>> * reviewed list of Airbnb's
production cherries and flagged
>>>>>> those
>>>>>>>>>>>>> as
>>>>>>>>>>>>>>> `Fix
>>>>>>>>>>>>>>>>> Version == 1.8.2`
>>>>>>>>>>>>>>>>> * Started branch v1-8-2.rc1
and started picking cherries
>>>> using
>>>>>>>>>>>>>>>>> `airflow-jira compare
1.8.2`
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> I'll finish going through
picking everything that targeted
>>>>>> 1.8.2
>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>> does
>>>>>>>>>>>>>>>>> not create merge conflict.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> If there's anything flagged
as "blocker" that generates
>> merge
>>>>>>>>>>>>>> conflict,
>>>>>>>>>>>>>>>>> I'll go case by case
about it.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Soon after, I should
be able to announce 1.8.2 RC1,
>> hopefully
>>>>>>>>>>>>>> sometime
>>>>>>>>>>>>>>>>> tomorrow or Friday.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Let me know if there's
anything else I'm missing that I
>>>> should
>>>>>>>>>>>>>>> consider.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Cheers!
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Max
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> 
>>>> 
>> 
>> 


Mime
View raw message