airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bolke de Bruin <bdbr...@gmail.com>
Subject Re: 1.8.1 release
Date Mon, 03 Apr 2017 21:19:13 GMT
Done.


> On 3 Apr 2017, at 22:22, Chris Riccomini <criccomini@apache.org> wrote:
> 
> Hey Bolke,
> 
> > Furthermore, do you have any timelines for getting to the release
> 
> Let's target April 17 to begin a vote for RC.
> 
> > Can I help in any way
> 
> I need people to review things. The more reviewers, the better.
> 
> Also, can you please reach a conclusion on whether the issues you raised (AIRFLOW-1019
and AIRFLOW-1013) are blockers or not (or just change the status unilaterally if people aren't
responding)?
> 
> Cheers,
> Chris
> 
> On Mon, Apr 3, 2017 at 1:06 PM, Bolke de Bruin <bdbruin@gmail.com <mailto:bdbruin@gmail.com>>
wrote:
> I have a PR out for 
> 
> * AIRFLOW-1001: https://github.com/apache/incubator-airflow/pull/2213 <https://github.com/apache/incubator-airflow/pull/2213>
> * AIRFLOW-1018: https://github.com/apache/incubator-airflow/pull/2212 <https://github.com/apache/incubator-airflow/pull/2212>
> 
> Furthermore, Sid, I checked AIRFLOW-1053 and while it is annoying I don’t think it
is a blocker: it happens only with @once dags that have a SLA, hardly very common. Nevertheless
a fix would be nice obviously.
> 
> Bolke
> 
>> On 3 Apr 2017, at 11:05, Bolke de Bruin <bdbruin@gmail.com <mailto:bdbruin@gmail.com>>
wrote:
>> 
>> Hey Chris,
>> 
>> AIRFLOW-1000 has a PR out. You might want to discuss it on the list what the impact
is though.
>> AIRFLOW-1018 I’ll have a PR out for shortly that allows “stdout” for the scheduler
log files. I did downgrade from blocker to critical.
>> AIRFLOW-719 Has a PR, including much needed increased test coverage, that I am pretty
sure is working, but needs verification (plz @Alex)
>> 
>> I would downgrade AIRFLOW-1019 to critical - especially as Dan is not around at the
moment.
>> 
>> Furthermore, do you have any timelines for getting to the release? Can I help in
any way? You might want to chase a couple of times ;-).
>> 
>> Bolke
>> 
>>> On 30 Mar 2017, at 19:48, siddharth anand <sanand@apache.org <mailto:sanand@apache.org>>
wrote:
>>> 
>>> Chris,
>>> I've submitted PRs for :
>>> 
>>>  - PR [AIRFLOW-1013] :
>>>  https://github.com/apache/incubator-airflow/pull/2203 <https://github.com/apache/incubator-airflow/pull/2203>
>>>  - PR [AIRFLOW-1054]:
>>>  https://github.com/apache/incubator-airflow/pull/2201 <https://github.com/apache/incubator-airflow/pull/2201>
>>> 
>>> And filed a blocker for a new issue. Essentially, @once DAGs cannot be
>>> created if catchup=False :
>>> https://issues.apache.org/jira/browse/AIRFLOW-1055 <https://issues.apache.org/jira/browse/AIRFLOW-1055>
>>> 
>>> I have a PR that works for this, but will need to add unit tests for it as
>>> well as for AIRFLOW-1013.
>>> 
>>> -s
>>> 
>>> On Wed, Mar 29, 2017 at 3:24 PM, siddharth anand <sanand@apache.org <mailto:sanand@apache.org>>
wrote:
>>> 
>>>> Didn't realize https://issues.apache.org/jira/browse/AIRFLOW-1013 <https://issues.apache.org/jira/browse/AIRFLOW-1013>
was a
>>>> blocker. I will have a PR shortly.
>>>> -s
>>>> 
>>>> On Wed, Mar 29, 2017 at 2:07 PM, Chris Riccomini <criccomini@apache.org
<mailto:criccomini@apache.org>>
>>>> wrote:
>>>> 
>>>>> The following three JIRAs were not merged into the v1-8-test branch,
but
>>>>> are listed as part of the 1.8.1 release:
>>>>> 
>>>>> AIRFLOW-1017 b2b9587cca9195229ab107394ad94b7702c70e37
>>>>> AIRFLOW-906 bc47200711be4d2c0b36b772651dae4f5e01a204
>>>>> AIRFLOW-858 94dc7fb0a6bb3c563d9df6566cd52a59bd0c4629
>>>>> AIRFLOW-832 b0ae70d3a8e935dc9266b6853683ae5375a7390b
>>>>> 
>>>>> I'm going to merge them in now.
>>>>> 
>>>>> On Wed, Mar 29, 2017 at 1:53 PM, Chris Riccomini <criccomini@apache.org
<mailto:criccomini@apache.org>>
>>>>> wrote:
>>>>> 
>>>>>> Hey Bolke,
>>>>>> 
>>>>>> Great. Assuming your PR is committed, that leaves five blockers:
>>>>>> 
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1000 <https://issues.apache.org/jira/browse/AIRFLOW-1000>
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1001 <https://issues.apache.org/jira/browse/AIRFLOW-1001>
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013 <https://issues.apache.org/jira/browse/AIRFLOW-1013>
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1018 <https://issues.apache.org/jira/browse/AIRFLOW-1018>
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1019 <https://issues.apache.org/jira/browse/AIRFLOW-1019>
>>>>>> 
>>>>>> I've also got a list of all open 1.8.1 JIRAs [1].
>>>>>> 
>>>>>> Cheers,
>>>>>> Chris
>>>>>> 
>>>>>> [1] https://issues.apache.org/jira/issues/?jql=project%20% <https://issues.apache.org/jira/issues/?jql=project%20%>
>>>>>> 3D%20AIRFLOW%20AND%20status%20in%20(Open%2C%20%22In%
>>>>>> 20Progress%22%2C%20Reopened)%20AND%20fixVersion%20%3D%201.8.1
>>>>>> 
>>>>>> On Mon, Mar 27, 2017 at 8:59 PM, Bolke de Bruin <bdbruin@gmail.com
<mailto:bdbruin@gmail.com>>
>>>>> wrote:
>>>>>> 
>>>>>>> Hi Chris,
>>>>>>> 
>>>>>>> I have a PR out for
>>>>>>> 
>>>>>>> * Revert of 719, which makes 982 obsolete and removes 983 from
the
>>>>>>> blockers list and just a new feature.
>>>>>>> 
>>>>>>> See: https://github.com/apache/incubator-airflow/pull/2195 <https://github.com/apache/incubator-airflow/pull/2195>
<
>>>>>>> https://github.com/apache/incubator-airflow/pull/2195 <https://github.com/apache/incubator-airflow/pull/2195>>
>>>>>>> 
>>>>>>> Cc: @alexvanboxel
>>>>>>> 
>>>>>>> Bolke
>>>>>>> 
>>>>>>>> On 24 Mar 2017, at 10:21, Chris Riccomini <criccomini@apache.org
<mailto:criccomini@apache.org>>
>>>>>>> wrote:
>>>>>>>> 
>>>>>>>> Hey all,
>>>>>>>> 
>>>>>>>> I've let this thread sit for a while. Here are a list of
the issues
>>>>> that
>>>>>>>> were raised:
>>>>>>>> 
>>>>>>>> BLOCKERS:
>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-982 <https://issues.apache.org/jira/browse/AIRFLOW-982>
>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-983 <https://issues.apache.org/jira/browse/AIRFLOW-983>
>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1019 <https://issues.apache.org/jira/browse/AIRFLOW-1019>
>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1017 <https://issues.apache.org/jira/browse/AIRFLOW-1017>
>>>>>>>> 
>>>>>>>> NICE TO HAVE:
>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1015 <https://issues.apache.org/jira/browse/AIRFLOW-1015>
>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013 <https://issues.apache.org/jira/browse/AIRFLOW-1013>
>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1004 <https://issues.apache.org/jira/browse/AIRFLOW-1004>
>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1003 <https://issues.apache.org/jira/browse/AIRFLOW-1003>
>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1001 <https://issues.apache.org/jira/browse/AIRFLOW-1001>
>>>>>>>> 
>>>>>>>> It looks like AIRFLOW-1017 is done, though the JIRA is not
closed.
>>>>>>>> 
>>>>>>>> The rest remain open. I will wait on the release until the
remaining
>>>>>>>> blockers are finished. Dan/Daniel, can you comment on status?
>>>>>>>> 
>>>>>>>> Ruslan, if you want to work on your nice to haves, and submit
>>>>> patches,
>>>>>>>> that's great, otherwise I don't believe they'll get fixed
as part of
>>>>>>> 1.8.1.
>>>>>>>> 
>>>>>>>> Cheers,
>>>>>>>> Chris
>>>>>>>> 
>>>>>>>> On Wed, Mar 22, 2017 at 9:19 AM, Ruslan Dautkhanov <
>>>>>>> dautkhanov@gmail.com <mailto:dautkhanov@gmail.com>>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Thank you Sid!
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> Best regards,
>>>>>>>>> Ruslan
>>>>>>>>> 
>>>>>>>>> On Wed, Mar 22, 2017 at 12:01 AM, siddharth anand <
>>>>> sanand@apache.org <mailto:sanand@apache.org>>
>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> Ruslan,
>>>>>>>>>> Thanks for sharing this list. I can pick a few up.
I agree we
>>>>> should
>>>>>>> aim
>>>>>>>>> to
>>>>>>>>>> get some of them into 1.8.1.
>>>>>>>>>> 
>>>>>>>>>> -s
>>>>>>>>>> 
>>>>>>>>>> On Tue, Mar 21, 2017 at 2:29 PM, Ruslan Dautkhanov
<
>>>>>>> dautkhanov@gmail.com <mailto:dautkhanov@gmail.com>
>>>>>>>>>> 
>>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> Some of the issues I ran into while testing 1.8rc5
:
>>>>>>>>>>> 
>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1015
<https://issues.apache.org/jira/browse/AIRFLOW-1015>
>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013
<https://issues.apache.org/jira/browse/AIRFLOW-1013>
>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1004
<https://issues.apache.org/jira/browse/AIRFLOW-1004>
>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1003
<https://issues.apache.org/jira/browse/AIRFLOW-1003>
>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1001
<https://issues.apache.org/jira/browse/AIRFLOW-1001>
>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1015
<https://issues.apache.org/jira/browse/AIRFLOW-1015>
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> It would be great to have at least some of them
fixed in 1.8.1.
>>>>>>>>>>> 
>>>>>>>>>>> Thank you.
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> --
>>>>>>>>>>> Ruslan Dautkhanov
>>>>>>>>>>> 
>>>>>>>>>>> On Tue, Mar 21, 2017 at 3:02 PM, Dan Davydov
<
>>>>> dan.davydov@airbnb.com <mailto:dan.davydov@airbnb.com>
>>>>>>> .
>>>>>>>>>>> invalid
>>>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> Here is my list for targeted 1.8.1 fixes:
>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-982
<https://issues.apache.org/jira/browse/AIRFLOW-982>
>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-983
<https://issues.apache.org/jira/browse/AIRFLOW-983>
>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1019
<https://issues.apache.org/jira/browse/AIRFLOW-1019> (and in
>>>>> general
>>>>>>>>> the
>>>>>>>>>>>> slow
>>>>>>>>>>>> startup time from this new logic of orphaned/reset
task)
>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1017
<https://issues.apache.org/jira/browse/AIRFLOW-1017> (which I will
>>>>>>>>>>> hopefully
>>>>>>>>>>>> have a fix out for soon just finishing up
tests)
>>>>>>>>>>>> 
>>>>>>>>>>>> We are also hitting a new issue with subdags
with rc5 that we
>>>>>>> weren't
>>>>>>>>>>>> hitting with rc4 where subdags will occasionally
just hang (had
>>>>> to
>>>>>>>>> roll
>>>>>>>>>>>> back from rc5 to rc4), I'll try to spin up
a JIRA for it soon
>>>>> which
>>>>>>>>>>> should
>>>>>>>>>>>> be on the list too.
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> On Tue, Mar 21, 2017 at 1:54 PM, Chris Riccomini
<
>>>>>>>>>> criccomini@apache.org <mailto:criccomini@apache.org>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> Agreed. I'm looking for a list of checksums/JIRAs
that we want
>>>>> in
>>>>>>>>> the
>>>>>>>>>>>>> bugfix release.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Tue, Mar 21, 2017 at 12:54 PM, Bolke
de Bruin <
>>>>>>>>> bdbruin@gmail.com <mailto:bdbruin@gmail.com>>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On 21 Mar 2017, at 12:51, Bolke
de Bruin <bdbruin@gmail.com <mailto:bdbruin@gmail.com>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> My suggestion, as we are using
semantic versioning is:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 1) no new features in the 1.8
branch
>>>>>>>>>>>>>>> 2) only bug fixes in the 1.8
branch
>>>>>>>>>>>>>>> 3) new features to land in 1.9
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> This allows companies to
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Have a "known" version and can move
to the new branch when they
>>>>>>>>>> want
>>>>>>>>>>> to
>>>>>>>>>>>>>> get new features. Obviously we only
support N-1, so when 1.10
>>>>>>>>> comes
>>>>>>>>>>> out
>>>>>>>>>>>>> we
>>>>>>>>>>>>>> stop supporting 1.8.X.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Sent from my iPhone
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On 21 Mar 2017, at 11:22,
Chris Riccomini <
>>>>>>>>>> criccomini@apache.org <mailto:criccomini@apache.org>>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Hey all,
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> I suggest that we start a
1.8.1 Airflow release now. The goal
>>>>>>>>>>> would
>>>>>>>>>>>>> be:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 1) get a second release under
our belt
>>>>>>>>>>>>>>>> 2) patch known issues with
the 1.8.0 release
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> I'm happy to run it, but
I saw Maxime mentioning that Airbnb
>>>>>>>>>> might
>>>>>>>>>>>>> want
>>>>>>>>>>>>>> to.
>>>>>>>>>>>>>>>> @Max et al, can you comment?
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Also, can folks supply JIRAs
for stuff that think needs to be
>>>>>>>>> in
>>>>>>>>>>> the
>>>>>>>>>>>>>> 1.8.1
>>>>>>>>>>>>>>>> bugfix release?
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>> Chris
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>>> 
>> 
> 
> 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message