airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bolke de Bruin <bdbr...@gmail.com>
Subject Re: 1.8.1 release
Date Wed, 05 Apr 2017 18:35:26 GMT
I have a PR out for this (AIRFLOW-1033). https://github.com/apache/incubator-airflow/pull/2220

 It seems large but that is due to activation of the ti_deps tests. See for the actual change:
https://github.com/apache/incubator-airflow/pull/2220/commits/e16678103bf7502e98689cd32f3636cbb67e492f#diff-09c898c9cd54512de2fb3a05bae9724c
<https://github.com/apache/incubator-airflow/pull/2220/commits/e16678103bf7502e98689cd32f3636cbb67e492f#diff-09c898c9cd54512de2fb3a05bae9724c>

Bolke

> On 5 Apr 2017, at 19:16, Chris Riccomini <criccomini@apache.org> wrote:
> 
> I've added it to 1.8.1 as a blocker.
> 
> On Wed, Apr 5, 2017 at 10:11 AM, Ruslan Dautkhanov <dautkhanov@gmail.com>
> wrote:
> 
>> https://issues.apache.org/jira/browse/AIRFLOW-1033
>> 
>> Is also 1.8 and not 1.8.0 blocker ?
>> 
>> Thanks.
>> 
>> 
>> 
>> --
>> Ruslan Dautkhanov
>> 
>> On Mon, Apr 3, 2017 at 3:20 PM, Bolke de Bruin <bdbruin@gmail.com> wrote:
>> 
>>> Hi Chris,
>>> 
>>> Please add:
>>> 
>>> https://issues.apache.org/jira/browse/AIRFLOW-1011 <
>>> https://issues.apache.org/jira/browse/AIRFLOW-1011> (PR:
>>> https://github.com/apache/incubator-airflow/pull/2179 <
>>> https://github.com/apache/incubator-airflow/pull/2179>)
>>> 
>>> To the blocker list. It was missed due to the “Airflow 1.8” affected
>>> version tag instead of “1.8.0”. I fixed that for you.
>>> 
>>> Bolke.
>>> 
>>>> On 3 Apr 2017, at 22:06, Bolke de Bruin <bdbruin@gmail.com> wrote:
>>>> 
>>>> I have a PR out for
>>>> 
>>>> * AIRFLOW-1001: https://github.com/apache/incubator-airflow/pull/2213
>> <
>>> https://github.com/apache/incubator-airflow/pull/2213>
>>>> * AIRFLOW-1018: https://github.com/apache/incubator-airflow/pull/2212
>> <
>>> https://github.com/apache/incubator-airflow/pull/2212>
>>>> 
>>>> Furthermore, Sid, I checked AIRFLOW-1053 and while it is annoying I
>>> don’t think it is a blocker: it happens only with @once dags that have a
>>> SLA, hardly very common. Nevertheless a fix would be nice obviously.
>>>> 
>>>> Bolke
>>>> 
>>>>> On 3 Apr 2017, at 11:05, Bolke de Bruin <bdbruin@gmail.com <mailto:
>>> bdbruin@gmail.com>> wrote:
>>>>> 
>>>>> Hey Chris,
>>>>> 
>>>>> AIRFLOW-1000 has a PR out. You might want to discuss it on the list
>>> what the impact is though.
>>>>> AIRFLOW-1018 I’ll have a PR out for shortly that allows “stdout”
for
>>> the scheduler log files. I did downgrade from blocker to critical.
>>>>> AIRFLOW-719 Has a PR, including much needed increased test coverage,
>>> that I am pretty sure is working, but needs verification (plz @Alex)
>>>>> 
>>>>> I would downgrade AIRFLOW-1019 to critical - especially as Dan is not
>>> around at the moment.
>>>>> 
>>>>> Furthermore, do you have any timelines for getting to the release? Can
>>> I help in any way? You might want to chase a couple of times ;-).
>>>>> 
>>>>> Bolke
>>>>> 
>>>>>> On 30 Mar 2017, at 19:48, siddharth anand <sanand@apache.org
>> <mailto:
>>> sanand@apache.org>> wrote:
>>>>>> 
>>>>>> Chris,
>>>>>> I've submitted PRs for :
>>>>>> 
>>>>>> - PR [AIRFLOW-1013] :
>>>>>> https://github.com/apache/incubator-airflow/pull/2203 <
>>> https://github.com/apache/incubator-airflow/pull/2203>
>>>>>> - PR [AIRFLOW-1054]:
>>>>>> https://github.com/apache/incubator-airflow/pull/2201 <
>>> https://github.com/apache/incubator-airflow/pull/2201>
>>>>>> 
>>>>>> And filed a blocker for a new issue. Essentially, @once DAGs cannot
>> be
>>>>>> created if catchup=False :
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1055 <
>>> https://issues.apache.org/jira/browse/AIRFLOW-1055>
>>>>>> 
>>>>>> I have a PR that works for this, but will need to add unit tests
for
>>> it as
>>>>>> well as for AIRFLOW-1013.
>>>>>> 
>>>>>> -s
>>>>>> 
>>>>>> On Wed, Mar 29, 2017 at 3:24 PM, siddharth anand <sanand@apache.org>
>>> wrote:
>>>>>> 
>>>>>>> Didn't realize https://issues.apache.org/jira/browse/AIRFLOW-1013
>>> was a
>>>>>>> blocker. I will have a PR shortly.
>>>>>>> -s
>>>>>>> 
>>>>>>> On Wed, Mar 29, 2017 at 2:07 PM, Chris Riccomini <
>>> criccomini@apache.org>
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> The following three JIRAs were not merged into the v1-8-test
>> branch,
>>> but
>>>>>>>> are listed as part of the 1.8.1 release:
>>>>>>>> 
>>>>>>>> AIRFLOW-1017 b2b9587cca9195229ab107394ad94b7702c70e37
>>>>>>>> AIRFLOW-906 bc47200711be4d2c0b36b772651dae4f5e01a204
>>>>>>>> AIRFLOW-858 94dc7fb0a6bb3c563d9df6566cd52a59bd0c4629
>>>>>>>> AIRFLOW-832 b0ae70d3a8e935dc9266b6853683ae5375a7390b
>>>>>>>> 
>>>>>>>> I'm going to merge them in now.
>>>>>>>> 
>>>>>>>> On Wed, Mar 29, 2017 at 1:53 PM, Chris Riccomini <
>>> criccomini@apache.org>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Hey Bolke,
>>>>>>>>> 
>>>>>>>>> Great. Assuming your PR is committed, that leaves five
blockers:
>>>>>>>>> 
>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1000
>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1001
>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013
>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1018
>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1019
>>>>>>>>> 
>>>>>>>>> I've also got a list of all open 1.8.1 JIRAs [1].
>>>>>>>>> 
>>>>>>>>> Cheers,
>>>>>>>>> Chris
>>>>>>>>> 
>>>>>>>>> [1] https://issues.apache.org/jira/issues/?jql=project%20%
>>>>>>>>> 3D%20AIRFLOW%20AND%20status%20in%20(Open%2C%20%22In%
>>>>>>>>> 20Progress%22%2C%20Reopened)%20AND%20fixVersion%20%3D%201.8.1
>>>>>>>>> 
>>>>>>>>> On Mon, Mar 27, 2017 at 8:59 PM, Bolke de Bruin <
>> bdbruin@gmail.com>
>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> Hi Chris,
>>>>>>>>>> 
>>>>>>>>>> I have a PR out for
>>>>>>>>>> 
>>>>>>>>>> * Revert of 719, which makes 982 obsolete and removes
983 from
>> the
>>>>>>>>>> blockers list and just a new feature.
>>>>>>>>>> 
>>>>>>>>>> See: https://github.com/apache/incubator-airflow/pull/2195
<
>>>>>>>>>> https://github.com/apache/incubator-airflow/pull/2195>
>>>>>>>>>> 
>>>>>>>>>> Cc: @alexvanboxel
>>>>>>>>>> 
>>>>>>>>>> Bolke
>>>>>>>>>> 
>>>>>>>>>>> On 24 Mar 2017, at 10:21, Chris Riccomini <
>> criccomini@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> Hey all,
>>>>>>>>>>> 
>>>>>>>>>>> I've let this thread sit for a while. Here are
a list of the
>>> issues
>>>>>>>> that
>>>>>>>>>>> were raised:
>>>>>>>>>>> 
>>>>>>>>>>> BLOCKERS:
>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-982
>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-983
>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1019
>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1017
>>>>>>>>>>> 
>>>>>>>>>>> NICE TO HAVE:
>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1015
>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013
>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1004
>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1003
>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1001
>>>>>>>>>>> 
>>>>>>>>>>> It looks like AIRFLOW-1017 is done, though the
JIRA is not
>> closed.
>>>>>>>>>>> 
>>>>>>>>>>> The rest remain open. I will wait on the release
until the
>>> remaining
>>>>>>>>>>> blockers are finished. Dan/Daniel, can you comment
on status?
>>>>>>>>>>> 
>>>>>>>>>>> Ruslan, if you want to work on your nice to haves,
and submit
>>>>>>>> patches,
>>>>>>>>>>> that's great, otherwise I don't believe they'll
get fixed as
>> part
>>> of
>>>>>>>>>> 1.8.1.
>>>>>>>>>>> 
>>>>>>>>>>> Cheers,
>>>>>>>>>>> Chris
>>>>>>>>>>> 
>>>>>>>>>>> On Wed, Mar 22, 2017 at 9:19 AM, Ruslan Dautkhanov
<
>>>>>>>>>> dautkhanov@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> Thank you Sid!
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> Best regards,
>>>>>>>>>>>> Ruslan
>>>>>>>>>>>> 
>>>>>>>>>>>> On Wed, Mar 22, 2017 at 12:01 AM, siddharth
anand <
>>>>>>>> sanand@apache.org>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> Ruslan,
>>>>>>>>>>>>> Thanks for sharing this list. I can pick
a few up. I agree we
>>>>>>>> should
>>>>>>>>>> aim
>>>>>>>>>>>> to
>>>>>>>>>>>>> get some of them into 1.8.1.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> -s
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Tue, Mar 21, 2017 at 2:29 PM, Ruslan
Dautkhanov <
>>>>>>>>>> dautkhanov@gmail.com
>>>>>>>>>>>>> 
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Some of the issues I ran into while
testing 1.8rc5 :
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1015
>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013
>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1004
>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1003
>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1001
>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1015
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> It would be great to have at least
some of them fixed in
>> 1.8.1.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Thank you.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Ruslan Dautkhanov
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Tue, Mar 21, 2017 at 3:02 PM,
Dan Davydov <
>>>>>>>> dan.davydov@airbnb.com
>>>>>>>>>> .
>>>>>>>>>>>>>> invalid
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Here is my list for targeted
1.8.1 fixes:
>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-982
>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-983
>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1019
(and in
>>>>>>>> general
>>>>>>>>>>>> the
>>>>>>>>>>>>>>> slow
>>>>>>>>>>>>>>> startup time from this new logic
of orphaned/reset task)
>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1017
(which I
>>> will
>>>>>>>>>>>>>> hopefully
>>>>>>>>>>>>>>> have a fix out for soon just
finishing up tests)
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> We are also hitting a new issue
with subdags with rc5 that
>> we
>>>>>>>>>> weren't
>>>>>>>>>>>>>>> hitting with rc4 where subdags
will occasionally just hang
>>> (had
>>>>>>>> to
>>>>>>>>>>>> roll
>>>>>>>>>>>>>>> back from rc5 to rc4), I'll try
to spin up a JIRA for it
>> soon
>>>>>>>> which
>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>> be on the list too.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Tue, Mar 21, 2017 at 1:54
PM, Chris Riccomini <
>>>>>>>>>>>>> criccomini@apache.org>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Agreed. I'm looking for a
list of checksums/JIRAs that we
>>> want
>>>>>>>> in
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> bugfix release.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Tue, Mar 21, 2017 at 12:54
PM, Bolke de Bruin <
>>>>>>>>>>>> bdbruin@gmail.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> On 21 Mar 2017, at
12:51, Bolke de Bruin <
>>> bdbruin@gmail.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> My suggestion, as
we are using semantic versioning is:
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 1) no new features
in the 1.8 branch
>>>>>>>>>>>>>>>>>> 2) only bug fixes
in the 1.8 branch
>>>>>>>>>>>>>>>>>> 3) new features to
land in 1.9
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> This allows companies
to
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Have a "known" version
and can move to the new branch when
>>> they
>>>>>>>>>>>>> want
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> get new features. Obviously
we only support N-1, so when
>>> 1.10
>>>>>>>>>>>> comes
>>>>>>>>>>>>>> out
>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>> stop supporting 1.8.X.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Sent from my iPhone
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> On 21 Mar 2017,
at 11:22, Chris Riccomini <
>>>>>>>>>>>>> criccomini@apache.org>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Hey all,
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> I suggest that
we start a 1.8.1 Airflow release now. The
>>> goal
>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>> be:
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 1) get a second
release under our belt
>>>>>>>>>>>>>>>>>>> 2) patch known
issues with the 1.8.0 release
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> I'm happy to
run it, but I saw Maxime mentioning that
>>> Airbnb
>>>>>>>>>>>>> might
>>>>>>>>>>>>>>>> want
>>>>>>>>>>>>>>>>> to.
>>>>>>>>>>>>>>>>>>> @Max et al, can
you comment?
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Also, can folks
supply JIRAs for stuff that think needs
>>> to be
>>>>>>>>>>>> in
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> 1.8.1
>>>>>>>>>>>>>>>>>>> bugfix release?
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>>>>> Chris
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>> 
>>>> 
>>> 
>>> 
>> 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message