airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Riccomini <criccom...@apache.org>
Subject Re: Airflow 1.9.0 status
Date Mon, 30 Oct 2017 18:51:36 GMT
I marked them as blocker. Still would like to understand how important this
is, though. The API is experimental, so I could see an argument for them
not being blocker provided the vulnerability is limited to experimental
areas.

On Mon, Oct 30, 2017 at 11:06 AM, Ash Berlin-Taylor <ash@firemirror.com>
wrote:

> I think we need to add
>
> https://issues.apache.org/jira/browse/AIRFLOW-1764
> https://issues.apache.org/jira/browse/AIRFLOW-1765
>
> to that list as critical security fixes. I'll tackled 1765 tomorrow (UK
> time) if no one gets round to it before then.
>
> -ash
>
> > On 30 Oct 2017, at 17:58, Chris Riccomini <criccomini@apache.org> wrote:
> >
> > Hey all,
> >
> > Current blockers for 1.9.0:
> >
> > https://issues.apache.org/jira/browse/AIRFLOW-1711
> > https://issues.apache.org/jira/browse/AIRFLOW-1018
> >
> > Cheers,
> > Chris
> >
> > On Tue, Oct 24, 2017 at 11:06 AM, Chris Riccomini <criccomini@apache.org
> >
> > wrote:
> >
> >> Hey all,
> >>
> >> Talked with @Bolke on Gitter. Here are the currently blocking issues for
> >> 1.9.0:
> >>
> >> AIRFLOW-1744      |Bug         ||Blocker   ||Open      |task.retries can
> >> be False
> >> AIRFLOW-1731      |Bug         ||Blocker   ||Open      |Import custom
> >> config on PYTHONPATH
> >> AIRFLOW-1641      |Bug         ||Blocker   ||In Progress|Task gets stuck
> >> in queued state
> >>
> >> PRs are out for them. After these three are merged, I'm planning to cut
> an
> >> RC.
> >>
> >> Cheers,
> >> Chris
> >>
> >> On Fri, Oct 20, 2017 at 7:49 AM, Bolke de Bruin <bdbruin@gmail.com>
> wrote:
> >>
> >>> I upgraded our production environment today. Some observations
> >>>
> >>> 1. Database migration (add max tries) didn’t workout of the box
> correctly
> >>> for Postgres. “max_tries = False” didn’t work and needed to be casted
> to an
> >>> int. Not sure how this gets to be false. This is a block imho
> >>> 2. SSHExecuteOperator not having backwards compatibility sucks. We
> needed
> >>> to rework quite a lot of dags
> >>> 3. We use LdAP for logins. We havent configured the ‘superuser’
> >>> ‘data_profiler’ groups and thus should have automatic superuser
> privileges.
> >>> We don’t at the moment (eg. we cannot manage connections or see the
> admin
> >>> screens).
> >>>
> >>> After fixing #1, the rest seems quite normal. Happy with the new
> logging.
> >>>
> >>> Bolke
> >>>
> >>>
> >>>> On 20 Oct 2017, at 12:16, Driesprong, Fokko <fokko@driesprong.frl>
> >>> wrote:
> >>>>
> >>>> Hi Bolke,
> >>>>
> >>>> This one is a blocker: https://issues.apache.org/jira
> >>> /browse/AIRFLOW-1731.
> >>>>
> >>>> This requirers that we are sure that the /plugin/ folder is on the
> path:
> >>>> https://github.com/apache/incubator-airflow/blob/eb2f589099b
> >>> 87743482c2eb16261b49e284dcd96/airflow/plugins_manager.py
> >>>>
> >>>> And maybe add an additional test and updating the docs a bit.
> Hopefully
> >>>> somewhere in the next few days, however I'm a bit busy with the Spark
> >>>> Summit.
> >>>>
> >>>> Cheers, Fokko
> >>>>
> >>>> 2017-10-19 21:03 GMT+02:00 Bolke de Bruin <bdbruin@gmail.com>:
> >>>>
> >>>>> Im planning to upgrade our production to 1.9.0alphaX tomorrow.
> >>>>>
> >>>>> What are the issues that are open?
> >>>>>
> >>>>> Cheers
> >>>>> Bolke
> >>>>>
> >>>>>> On 9 Oct 2017, at 20:24, Chris Riccomini <criccomini@apache.org>
> >>> wrote:
> >>>>>>
> >>>>>> K, I will plan on cutting an alpha1 later this week. Something for
> you
> >>>>> guys to play with.
> >>>>>>
> >>>>>> On Mon, Oct 9, 2017 at 11:23 AM, Bolke de Bruin <bdbruin@gmail.com
> >>>>> <mailto:bdbruin@gmail.com>> wrote:
> >>>>>> hi Chris,
> >>>>>>
> >>>>>> We are still running pre alpha. I am a bit preoccupied with
> preparing
> >>>>> for a conference and the team is readying a release of one of our
> core
> >>>>> products. So it will probably will be after this week when I get my
> >>> hands
> >>>>> dirty again.
> >>>>>>
> >>>>>> Cheers
> >>>>>> Bolke
> >>>>>>
> >>>>>> Verstuurd vanaf mijn iPad
> >>>>>>
> >>>>>>> Op 4 okt. 2017 om 23:35 heeft Chris Riccomini <
> criccomini@apache.org
> >>>>> <mailto:criccomini@apache.org>> het volgende geschreven:
> >>>>>>>
> >>>>>>> Hey all,
> >>>>>>>
> >>>>>>> Checking in. Has anyone tested this in their environments? Stable
> >>>>> releases
> >>>>>>> depend on community involvement.
> >>>>>>>
> >>>>>>> Cheers,
> >>>>>>> Chris
> >>>>>>>
> >>>>>>> On Tue, Oct 3, 2017 at 10:30 PM, Chris Riccomini <
> >>>>> criccomini@apache.org <mailto:criccomini@apache.org>>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> I've added AIRFLOW-1635 to the v1-9-test branch. It's not in
> alpha0,
> >>>>> but
> >>>>>>>> will be included in alpha1.
> >>>>>>>>
> >>>>>>>>> On Tue, Oct 3, 2017 at 4:13 PM, Feng Lu
> <fenglu@google.com.invalid
> >>>>
> >>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> Hi Chris,
> >>>>>>>>>
> >>>>>>>>> I know it's annoying to have last minute commit com in, but this
> >>> is a
> >>>>>>>>> highly desirable feature for folks using GCP operators, is it
> >>>>> possible to
> >>>>>>>>> include AIRFLOW-1635
> >>>>>>>>> <https://github.com/apache/incubator-airflow/commit/b3e985a3 <
> >>>>> https://github.com/apache/incubator-airflow/commit/b3e985a3>
> >>>>>>>>> 146272ecfd3ceaaa0d8567e4e9e117d4>
> >>>>>>>>> in?
> >>>>>>>>> More than happy to offer help if there's something I can do.
> >>>>>>>>> Thanks a lot.
> >>>>>>>>>
> >>>>>>>>> Feng
> >>>>>>>>>
> >>>>>>>>> On Mon, Oct 2, 2017 at 3:24 PM, Chris Riccomini <
> >>>>> criccomini@apache.org <mailto:criccomini@apache.org>>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Hey all,
> >>>>>>>>>>
> >>>>>>>>>> I have cut a 1.9.0alpha0 release of Airflow. You can download it
> >>>>> here:
> >>>>>>>>>>
> >>>>>>>>>> https://dist.apache.org/repos/dist/dev/incubator/airflow/1.
> >>>>> 9.0alpha0/ <https://dist.apache.org/repos/dist/dev/incubator/
> >>>>> airflow/1.9.0alpha0/>
> >>>>>>>>>>
> >>>>>>>>>> The bin tarball can be installed with pip:
> >>>>>>>>>>
> >>>>>>>>>> pip install apache-airflow-1.9.0alpha0+incubating-bin.tar.gz
> >>>>>>>>>>
> >>>>>>>>>> The goal is to have the community install and run this to expose
> >>> any
> >>>>>>>>> bugs
> >>>>>>>>>> before we move on to official release candidates.
> >>>>>>>>>>
> >>>>>>>>>> Here are the outstanding blocker bugs for 1.9.0:
> >>>>>>>>>>
> >>>>>>>>>> AIRFLOW-1525 |Improvement |Fix minor LICENSE & NOTICE issue
> >>>>>>>>>> AIRFLOW-1258 |Bug         |TaskInstances within SubDagOperator
> are
> >>>>>>>>> marked
> >>>>>>>>>> as
> >>>>>>>>>> AIRFLOW-1055 |Bug         |airflow/jobs.py:create_dag_run()
> >>>>> exception
> >>>>>>>>>> for
> >>>>>>>>>> @on
> >>>>>>>>>> AIRFLOW-1018 |Bug         |Scheduler DAG processes can not log
> to
> >>>>>>>>> stdout
> >>>>>>>>>> AIRFLOW-1013 |Bug         |airflow/jobs.py:manage_slas()
> >>> exception
> >>>>>>>>> for
> >>>>>>>>>> @once
> >>>>>>>>>> AIRFLOW-976  |Bug         |Mark success running task causes it
> to
> >>>>> fail
> >>>>>>>>>>
> >>>>>>>>>> Cheers,
> >>>>>>>>>> Chris
> >>>>>>>>>>
> >>>>>>>>>> On Fri, Sep 29, 2017 at 3:54 PM, Chris Riccomini <
> >>>>> criccomini@apache.org <mailto:criccomini@apache.org>
> >>>>>>>>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Welp. Work got in the way, so I'll cut the beta on Monday. :)
> >>>>>>>>>>>
> >>>>>>>>>>> On Thu, Sep 28, 2017 at 1:32 PM, Chris Riccomini <
> >>>>>>>>> criccomini@apache.org <mailto:criccomini@apache.org>>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Works for me. Will try and cut a beta before end of week.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Blockers for 1.9.0 are:
> >>>>>>>>>>>>
> >>>>>>>>>>>> AIRFLOW-1611 |Bug         |Customize logging in Airflow
> >>>>>>>>>>>> AIRFLOW-1525 |Improvement |Fix minor LICENSE & NOTICE issue
> >>>>>>>>>>>> AIRFLOW-1258 |Bug         |TaskInstances within SubDagOperator
> >>> are
> >>>>>>>>>> marked
> >>>>>>>>>>>> as
> >>>>>>>>>>>> AIRFLOW-976  |Bug         |Mark success running task causes it
> >>> to
> >>>>>>>>> fail
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Thu, Sep 28, 2017 at 1:09 PM, Bolke de Bruin <
> >>>>> bdbruin@gmail.com <mailto:bdbruin@gmail.com>>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Hi Chris
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Can I suggest releasing a beta? The stable branch is only cut
> >>> at
> >>>>> RC
> >>>>>>>>>>>>> time. Betas allow us a broader exposure. It also gives us a
> >>>>> point of
> >>>>>>>>>>>>> reference.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> In addition the list below are mostly longer standing issues
> >>> that
> >>>>>>>>> are
> >>>>>>>>>>>>> also part of the 1.8.x branch. Maybe only consider 1611,
> 1525,
> >>>>> 1258,
> >>>>>>>>>> and
> >>>>>>>>>>>>> 976 as blocker?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Cheers
> >>>>>>>>>>>>> Bolke
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Verstuurd vanaf mijn iPad
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Op 28 sep. 2017 om 19:49 heeft Chris Riccomini <
> >>>>>>>>>> criccomini@apache.org <mailto:criccomini@apache.org>>
> >>>>>>>>>>>>> het volgende geschreven:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hey all,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I was planning to cut a 1.9.0 stable branch and 1.9.0 beta
> >>>>>>>>> release,
> >>>>>>>>>> but
> >>>>>>>>>>>>>> seeing as there are several outstanding bugs, I'm going to
> >>>>> delay.
> >>>>>>>>>> Here
> >>>>>>>>>>>>> are
> >>>>>>>>>>>>>> the bugs that I'm tracking:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> AIRFLOW-1611 |Bug         |Customize logging in Airflow
> >>>>>>>>>>>>>> AIRFLOW-1525 |Improvement |Fix minor LICENSE & NOTICE issue
> >>>>>>>>>>>>>> AIRFLOW-1258 |Bug         |TaskInstances within
> SubDagOperator
> >>>>> are
> >>>>>>>>>>>>> marked as
> >>>>>>>>>>>>>> AIRFLOW-1055 |Bug         |airflow/jobs.py:create_dag_run()
> >>>>>>>>>> exception
> >>>>>>>>>>>>> for
> >>>>>>>>>>>>>> @on
> >>>>>>>>>>>>>> AIRFLOW-1018 |Bug         |Scheduler DAG processes can not
> log
> >>>>> to
> >>>>>>>>>>>>> stdout
> >>>>>>>>>>>>>> AIRFLOW-1013 |Bug         |airflow/jobs.py:manage_slas()
> >>>>>>>>> exception
> >>>>>>>>>>>>> for @once
> >>>>>>>>>>>>>> AIRFLOW-988  |Bug         |SLA Miss Callbacks Are Repeated
> if
> >>>>>>>>> Email
> >>>>>>>>>> is
> >>>>>>>>>>>>> Not
> >>>>>>>>>>>>>> be
> >>>>>>>>>>>>>> AIRFLOW-976  |Bug         |Mark success running task causes
> it
> >>>>> to
> >>>>>>>>>> fail
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> These are the priority issues. Once they're merged, I'll cut
> >>> the
> >>>>>>>>>>>>>> v1-9-stable and beta release.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> If you can help clean this up, that would be really
> >>> appreciated.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Cheers,
> >>>>>>>>>>>>>> Chris
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Thu, Sep 28, 2017 at 10:06 AM, Chris Riccomini <
> >>>>>>>>>>>>> criccomini@apache.org <mailto:criccomini@apache.org>>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Marked it for 1.9.0.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Thu, Sep 28, 2017 at 9:56 AM, Charlie Jones <
> >>>>>>>>> cjones@simpli.fi <mailto:cjones@simpli.fi>>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Is there any chance we could include AIRFLOW-988 in 1.9.0?
> >>> SLA
> >>>>>>>>>>>>> callbacks
> >>>>>>>>>>>>>>>> are not working correctly without emails... Its not a
> major
> >>>>> bug,
> >>>>>>>>>> but
> >>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>> does cause us some annoyance in our current deployment.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Link to Jira:
> >>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-988 <
> >>>>> https://issues.apache.org/jira/browse/AIRFLOW-988>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Link to PR:
> >>>>>>>>>>>>>>>> https://github.com/apache/incubator-airflow/pull/2415 <
> >>>>> https://github.com/apache/incubator-airflow/pull/2415>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>> Charlie Jones
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> CHARLIE JONES
> >>>>>>>>>>>>>>>> Data Engineer
> >>>>>>>>>>>>>>>> cjones@simpli.fi <mailto:cjones@simpli.fi>  |  M:
> >>>>> 972.821.7631 <tel:972.821.7631>
> >>>>>>>>>>>>>>>> __________________________________________________
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Programmatic Performance.* Localized.*
> >>>>>>>>>>>>>>>> __________________________________________________
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> 1407 Texas Street  |  Suite 202  |  Fort Worth, TX 76102
> >>>>>>>>>>>>>>>> 800.840.0768 <tel:800.840.0768>  |  www.simpli.fi <
> >>>>> http://www.simpli.fi/>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Wed, Sep 27, 2017 at 12:11 PM, Chris Riccomini <
> >>>>>>>>>>>>> criccomini@apache.org <mailto:criccomini@apache.org>>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Merged.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Tue, Sep 26, 2017 at 2:25 PM, Ryan Buckley <
> >>>>>>>>>>>>>>>> ryan.buckley@bluecore.com <mailto:ryan.buckley@bluecore.
> com
> >>>>>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Would it be possible to include AIRFLOW-1587?
> >>>>>>>>>>>>>>>>>> Running dags from the UI is currently broken on the
> 1.9.0
> >>>>>>>>> branch
> >>>>>>>>>>>>> due
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>> this issue.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> https://github.com/apache/incubator-airflow/pull/2590 <
> >>>>> https://github.com/apache/incubator-airflow/pull/2590>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>> Ryan
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Tue, Sep 26, 2017 at 8:14 AM, Driesprong, Fokko
> >>>>>>>>>>>>>>>> <fokko@driesprong.frl
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I would like to include AIRFLOW-1611 in the 1.9.0
> >>> release:
> >>>>>>>>>>>>>>>>>>> https://github.com/apache/incubator-airflow/pull/2631
> <
> >>>>> https://github.com/apache/incubator-airflow/pull/2631>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Currently importing a custom logging configuration is
> not
> >>>>>>>>> work
> >>>>>>>>>> (as
> >>>>>>>>>>>>>>>> far
> >>>>>>>>>>>>>>>>>> as I
> >>>>>>>>>>>>>>>>>>> know). Any feedback on the PR would also be
> appreciated.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Cheers, Fokko
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 2017-09-25 23:27 GMT+02:00 Chris Riccomini <
> >>>>>>>>>> criccomini@apache.org <mailto:criccomini@apache.org>
> >>>>>>>>>>>>>> :
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Done!
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Mon, Sep 25, 2017 at 5:11 AM, Michael Crawford <
> >>>>>>>>>>>>>>>>>>>> michael.crawford@modernizingmedicine.com <mailto:
> >>>>> michael.crawford@modernizingmedicine.com>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Can you slide the aws and emr connection type fix in?
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1636 <
> >>>>> https://issues.apache.org/jira/browse/AIRFLOW-1636> <
> >>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1636 <
> >>>>> https://issues.apache.org/jira/browse/AIRFLOW-1636>>
> >>>>>>>>>>>>>>>>>>>>> https://github.com/apache/
> incubator-airflow/pull/2626
> >>> <
> >>>>> https://github.com/apache/incubator-airflow/pull/2626> <
> >>>>>>>>>>>>>>>>>>>>> https://github.com/apache/
> incubator-airflow/pull/2626
> >>> <
> >>>>> https://github.com/apache/incubator-airflow/pull/2626>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> It keeps the connection type from getting blanked out
> >>> on
> >>>>>>>>> edit
> >>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>> these
> >>>>>>>>>>>>>>>>>>>>> types.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Sep 21, 2017, at 1:27 PM, Chris Riccomini <
> >>>>>>>>>>>>>>>>>> criccomini@apache.org <mailto:criccomini@apache.org>>
> >>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Absolutely. Just cherry-picked. I've been looking
> >>>>> forward
> >>>>>>>>> to
> >>>>>>>>>>>>>>>>> these
> >>>>>>>>>>>>>>>>>>>> fixes!
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Wed, Sep 20, 2017 at 5:23 PM, Alex Guziel <
> >>>>>>>>>>>>>>>>>> alex.guziel@airbnb.com <mailto:alex.guziel@airbnb.com>
> >>>>>>>>>>>>>>>>>>> .
> >>>>>>>>>>>>>>>>>>>>> invalid
> >>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Can we get this in?
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1519
> <
> >>>>> https://issues.apache.org/jira/browse/AIRFLOW-1519>
> >>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1621
> <
> >>>>> https://issues.apache.org/jira/browse/AIRFLOW-1621>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> https://github.com/apache/
> incubator-airflow/commit/
> >>> <
> >>>>> https://github.com/apache/incubator-airflow/commit/>
> >>>>>>>>>>>>>>>>>>>>>>> b6d2e0a46978e93e16576604624f57d1388814f2
> >>>>>>>>>>>>>>>>>>>>>>> https://github.com/apache/
> incubator-airflow/commit/
> >>> <
> >>>>> https://github.com/apache/incubator-airflow/commit/>
> >>>>>>>>>>>>>>>>>>>>>>> 656d045e90bf67ca484a3778b2a07a419bfb324a
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> It speeds up loading times a lot, so it's a good
> >>> thing
> >>>>> to
> >>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>> 1.9.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Wed, Sep 20, 2017 at 11:14 AM, Chris Riccomini <
> >>>>>>>>>>>>>>>>>>>>> criccomini@apache.org <mailto:criccomini@apache.org>
> >
> >>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Sounds good. I'll plan on stable+beta next week,
> >>> then.
> >>>>>>>>>>>>>>>> Initial
> >>>>>>>>>>>>>>>>>>>> warning
> >>>>>>>>>>>>>>>>>>>>>>>> stands, that I will start locking down what can
> get
> >>>>> into
> >>>>>>>>>>>>>>>> 1.9.0
> >>>>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>> point.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Wed, Sep 20, 2017 at 11:10 AM, Bolke de Bruin <
> >>>>>>>>>>>>>>>>>>> bdbruin@gmail.com <mailto:bdbruin@gmail.com>>
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> No vote indeed, just to gather feedback on a
> >>>>> particular
> >>>>>>>>>>>>>>>> fixed
> >>>>>>>>>>>>>>>>>>> point
> >>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>> time. It also gives a bit more trust to a tarball
> >>>>> than
> >>>>>>>>> to
> >>>>>>>>>> a
> >>>>>>>>>>>>>>>>> git
> >>>>>>>>>>>>>>>>>>>> pull.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Bolke
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> On 20 Sep 2017, at 20:09, Chris Riccomini <
> >>>>>>>>>>>>>>>>>> criccomini@apache.org <mailto:criccomini@apache.org>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> I can do a beta. Is the process significantly
> >>>>>>>>> different?
> >>>>>>>>>>>>>>>>> IIRC,
> >>>>>>>>>>>>>>>>>>> it's
> >>>>>>>>>>>>>>>>>>>>>>>>>> basically the same, just no vote, right?
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Sep 20, 2017 at 10:56 AM, Bolke de
> Bruin <
> >>>>>>>>>>>>>>>>>>>> bdbruin@gmail.com <mailto:bdbruin@gmail.com>>
> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Are you sure you want to go ahead and do RCs
> >>> right
> >>>>>>>>> away?
> >>>>>>>>>>>>>>>>>> Isn’t a
> >>>>>>>>>>>>>>>>>>>>>>> beta
> >>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>> bit smarter?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> - Bolke
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On 20 Sep 2017, at 19:41, Chris Riccomini <
> >>>>>>>>>>>>>>>>>>> criccomini@apache.org <mailto:criccomini@apache.org>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Hey all,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I want to send out a warning that I'm planning
> >>> to
> >>>>>>>>> cut
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> stable
> >>>>>>>>>>>>>>>>>>>>>>>> branch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> next week, and begin the RC1 release vote.
> Once
> >>>>> the
> >>>>>>>>>>>>>>>> stable
> >>>>>>>>>>>>>>>>>>> branch
> >>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>> cut, I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> will be locking down what commits get cherry
> >>>>> picked
> >>>>>>>>>> into
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>> branch,
> >>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> will only be doing PRs that are required to
> get
> >>>>> the
> >>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>> out.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Cheers,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Chris
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Sep 18, 2017 at 11:19 AM, Chris
> >>> Riccomini
> >>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>> criccomini@apache.org <mailto:
> >>> criccomini@apache.org>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hey all,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> An update on the 1.9.0 release. Here are the
> >>>>>>>>>>>>>>>> outstanding
> >>>>>>>>>>>>>>>>> PRs
> >>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> slated to be included into 1.9.0:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISSUE ID     |STATUS    |DESCRIPTION
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1617 |Open      |XSS Vulnerability in
> >>>>>>>>> Variable
> >>>>>>>>>>>>>>>>>>> endpoint
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1611 |Open      |Customize logging in
> >>>>>>>>> Airflow
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1605 |Reopened  |Fix log source of
> >>> local
> >>>>>>>>>>>>>>>> loggers
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1604 |Open      |Rename the logger to
> >>> log
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1525 |Open      |Fix minor LICENSE &
> >>>>> NOTICE
> >>>>>>>>>>>>>>>> issue
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1499 |In Progres|Eliminate duplicate
> >>> and
> >>>>>>>>>>>>>>>> unneeded
> >>>>>>>>>>>>>>>>>> code
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1198 |Open      |HDFSOperator to
> >>> operate
> >>>>>>>>> HDFS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1055 |Open
> >>>>>>>>> |airflow/jobs.py:create_dag_ru
> >>>>>>>>>>>>>>>> n()
> >>>>>>>>>>>>>>>>>>>>>>> exception
> >>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> @on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1019 |Open      |active_dagruns
> >>> shouldn't
> >>>>>>>>>>>>>>>> include
> >>>>>>>>>>>>>>>>>>> paused
> >>>>>>>>>>>>>>>>>>>>>>>> DAGs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1018 |Open      |Scheduler DAG
> >>> processes
> >>>>>>>>> can
> >>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>> log
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> stdout
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1015 |Open      |TreeView displayed
> >>> over
> >>>>>>>>> task
> >>>>>>>>>>>>>>>>>>> instances
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-1013 |Open
> >>>>>>>>> |airflow/jobs.py:manage_slas()
> >>>>>>>>>>>>>>>>>>> exception
> >>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> @once
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-976  |Open      |Mark success running
> >>>>> task
> >>>>>>>>>>>>>>>> causes
> >>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>> fail
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-914  |Open      |Refactor
> >>>>>>>>>>>>>>>>>>> BackfillJobTest.test_backfill_
> >>>>>>>>>>>>>>>>>>>>>>>>>>> examples
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-913  |Open      |Refactor
> >>>>>>>>>>>>>>>>>>> tests.CoreTest.test_scheduler_
> >>>>>>>>>>>>>>>>>>>>>>> job
> >>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> real
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-912  |Open      |Refactor tests and
> >>> build
> >>>>>>>>>>>>>>>> matrix
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-888  |Open      |Operators should not
> >>>>> push
> >>>>>>>>>>>>>>>> XComs
> >>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>> default
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-828  |Open      |Add maximum size for
> >>>>> XComs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-825  |Open      |Add Dataflow
> semantics
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> AIRFLOW-788  |Open      |Context unexpectedly
> >>>>>>>>> added to
> >>>>>>>>>>>>>>>>> hive
> >>>>>>>>>>>>>>>>>>> conf
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will be locking down what can get
> >>> cherry-picked
> >>>>>>>>> into
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> 1.9.0
> >>>>>>>>>>>>>>>>>>>>>>>>> branch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> shortly, so if you have something you want
> in,
> >>>>>>>>> please
> >>>>>>>>>>>>>>>> set
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> fix
> >>>>>>>>>>>>>>>>>>>>>>>>>>> version
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> to 1.9.0.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> We (at WePay) have deployed 1.9.0 into our
> dev
> >>>>>>>>>> cluster,
> >>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>> has
> >>>>>>>>>>>>>>>>>>>>>>>>> been
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> running smoothly for several days.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ** I could really use help verifying
> stability.
> >>>>> If
> >>>>>>>>> you
> >>>>>>>>>>>>>>>> run
> >>>>>>>>>>>>>>>>>>>>>>> Airflow,
> >>>>>>>>>>>>>>>>>>>>>>>>> it's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> in your best interest to deploy the 1.9.0
> test
> >>>>>>>>> branch
> >>>>>>>>>>>>>>>>>>> somewhere,
> >>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>> verify
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> it's working for your workload. **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Cheers,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Chris
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>
> >>>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message