airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Riccomini <criccom...@apache.org>
Subject Re: Airflow 1.9.0 status
Date Tue, 24 Oct 2017 18:06:22 GMT
Hey all,

Talked with @Bolke on Gitter. Here are the currently blocking issues for
1.9.0:

AIRFLOW-1744      |Bug         ||Blocker   ||Open      |task.retries can be
False
AIRFLOW-1731      |Bug         ||Blocker   ||Open      |Import custom
config on PYTHONPATH
AIRFLOW-1641      |Bug         ||Blocker   ||In Progress|Task gets stuck in
queued state

PRs are out for them. After these three are merged, I'm planning to cut an
RC.

Cheers,
Chris

On Fri, Oct 20, 2017 at 7:49 AM, Bolke de Bruin <bdbruin@gmail.com> wrote:

> I upgraded our production environment today. Some observations
>
> 1. Database migration (add max tries) didn’t workout of the box correctly
> for Postgres. “max_tries = False” didn’t work and needed to be casted to an
> int. Not sure how this gets to be false. This is a block imho
> 2. SSHExecuteOperator not having backwards compatibility sucks. We needed
> to rework quite a lot of dags
> 3. We use LdAP for logins. We havent configured the ‘superuser’
> ‘data_profiler’ groups and thus should have automatic superuser privileges.
> We don’t at the moment (eg. we cannot manage connections or see the admin
> screens).
>
> After fixing #1, the rest seems quite normal. Happy with the new logging.
>
> Bolke
>
>
> > On 20 Oct 2017, at 12:16, Driesprong, Fokko <fokko@driesprong.frl>
> wrote:
> >
> > Hi Bolke,
> >
> > This one is a blocker: https://issues.apache.org/
> jira/browse/AIRFLOW-1731.
> >
> > This requirers that we are sure that the /plugin/ folder is on the path:
> > https://github.com/apache/incubator-airflow/blob/
> eb2f589099b87743482c2eb16261b49e284dcd96/airflow/plugins_manager.py
> >
> > And maybe add an additional test and updating the docs a bit. Hopefully
> > somewhere in the next few days, however I'm a bit busy with the Spark
> > Summit.
> >
> > Cheers, Fokko
> >
> > 2017-10-19 21:03 GMT+02:00 Bolke de Bruin <bdbruin@gmail.com>:
> >
> >> Im planning to upgrade our production to 1.9.0alphaX tomorrow.
> >>
> >> What are the issues that are open?
> >>
> >> Cheers
> >> Bolke
> >>
> >>> On 9 Oct 2017, at 20:24, Chris Riccomini <criccomini@apache.org>
> wrote:
> >>>
> >>> K, I will plan on cutting an alpha1 later this week. Something for you
> >> guys to play with.
> >>>
> >>> On Mon, Oct 9, 2017 at 11:23 AM, Bolke de Bruin <bdbruin@gmail.com
> >> <mailto:bdbruin@gmail.com>> wrote:
> >>> hi Chris,
> >>>
> >>> We are still running pre alpha. I am a bit preoccupied with preparing
> >> for a conference and the team is readying a release of one of our core
> >> products. So it will probably will be after this week when I get my
> hands
> >> dirty again.
> >>>
> >>> Cheers
> >>> Bolke
> >>>
> >>> Verstuurd vanaf mijn iPad
> >>>
> >>>> Op 4 okt. 2017 om 23:35 heeft Chris Riccomini <criccomini@apache.org
> >> <mailto:criccomini@apache.org>> het volgende geschreven:
> >>>>
> >>>> Hey all,
> >>>>
> >>>> Checking in. Has anyone tested this in their environments? Stable
> >> releases
> >>>> depend on community involvement.
> >>>>
> >>>> Cheers,
> >>>> Chris
> >>>>
> >>>> On Tue, Oct 3, 2017 at 10:30 PM, Chris Riccomini <
> >> criccomini@apache.org <mailto:criccomini@apache.org>>
> >>>> wrote:
> >>>>
> >>>>> I've added AIRFLOW-1635 to the v1-9-test branch. It's not in alpha0,
> >> but
> >>>>> will be included in alpha1.
> >>>>>
> >>>>>> On Tue, Oct 3, 2017 at 4:13 PM, Feng Lu <fenglu@google.com.invalid>
> >> wrote:
> >>>>>>
> >>>>>> Hi Chris,
> >>>>>>
> >>>>>> I know it's annoying to have last minute commit com in, but
this is
> a
> >>>>>> highly desirable feature for folks using GCP operators, is it
> >> possible to
> >>>>>> include AIRFLOW-1635
> >>>>>> <https://github.com/apache/incubator-airflow/commit/b3e985a3
<
> >> https://github.com/apache/incubator-airflow/commit/b3e985a3>
> >>>>>> 146272ecfd3ceaaa0d8567e4e9e117d4>
> >>>>>> in?
> >>>>>> More than happy to offer help if there's something I can do.
> >>>>>> Thanks a lot.
> >>>>>>
> >>>>>> Feng
> >>>>>>
> >>>>>> On Mon, Oct 2, 2017 at 3:24 PM, Chris Riccomini <
> >> criccomini@apache.org <mailto:criccomini@apache.org>>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Hey all,
> >>>>>>>
> >>>>>>> I have cut a 1.9.0alpha0 release of Airflow. You can download
it
> >> here:
> >>>>>>>
> >>>>>>> https://dist.apache.org/repos/dist/dev/incubator/airflow/1.
> >> 9.0alpha0/ <https://dist.apache.org/repos/dist/dev/incubator/
> >> airflow/1.9.0alpha0/>
> >>>>>>>
> >>>>>>> The bin tarball can be installed with pip:
> >>>>>>>
> >>>>>>> pip install apache-airflow-1.9.0alpha0+incubating-bin.tar.gz
> >>>>>>>
> >>>>>>> The goal is to have the community install and run this to
expose
> any
> >>>>>> bugs
> >>>>>>> before we move on to official release candidates.
> >>>>>>>
> >>>>>>> Here are the outstanding blocker bugs for 1.9.0:
> >>>>>>>
> >>>>>>> AIRFLOW-1525 |Improvement |Fix minor LICENSE & NOTICE
issue
> >>>>>>> AIRFLOW-1258 |Bug         |TaskInstances within SubDagOperator
are
> >>>>>> marked
> >>>>>>> as
> >>>>>>> AIRFLOW-1055 |Bug         |airflow/jobs.py:create_dag_run()
> >> exception
> >>>>>>> for
> >>>>>>> @on
> >>>>>>> AIRFLOW-1018 |Bug         |Scheduler DAG processes can not
log to
> >>>>>> stdout
> >>>>>>> AIRFLOW-1013 |Bug         |airflow/jobs.py:manage_slas()
exception
> >>>>>> for
> >>>>>>> @once
> >>>>>>> AIRFLOW-976  |Bug         |Mark success running task causes
it to
> >> fail
> >>>>>>>
> >>>>>>> Cheers,
> >>>>>>> Chris
> >>>>>>>
> >>>>>>> On Fri, Sep 29, 2017 at 3:54 PM, Chris Riccomini <
> >> criccomini@apache.org <mailto:criccomini@apache.org>
> >>>>>>>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Welp. Work got in the way, so I'll cut the beta on Monday.
:)
> >>>>>>>>
> >>>>>>>> On Thu, Sep 28, 2017 at 1:32 PM, Chris Riccomini <
> >>>>>> criccomini@apache.org <mailto:criccomini@apache.org>>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Works for me. Will try and cut a beta before end
of week.
> >>>>>>>>>
> >>>>>>>>> Blockers for 1.9.0 are:
> >>>>>>>>>
> >>>>>>>>> AIRFLOW-1611 |Bug         |Customize logging in
Airflow
> >>>>>>>>> AIRFLOW-1525 |Improvement |Fix minor LICENSE &
NOTICE issue
> >>>>>>>>> AIRFLOW-1258 |Bug         |TaskInstances within
SubDagOperator
> are
> >>>>>>> marked
> >>>>>>>>> as
> >>>>>>>>> AIRFLOW-976  |Bug         |Mark success running
task causes it to
> >>>>>> fail
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Thu, Sep 28, 2017 at 1:09 PM, Bolke de Bruin
<
> >> bdbruin@gmail.com <mailto:bdbruin@gmail.com>>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Hi Chris
> >>>>>>>>>>
> >>>>>>>>>> Can I suggest releasing a beta? The stable branch
is only cut at
> >> RC
> >>>>>>>>>> time. Betas allow us a broader exposure. It
also gives us a
> >> point of
> >>>>>>>>>> reference.
> >>>>>>>>>>
> >>>>>>>>>> In addition the list below are mostly longer
standing issues
> that
> >>>>>> are
> >>>>>>>>>> also part of the 1.8.x branch. Maybe only consider
1611, 1525,
> >> 1258,
> >>>>>>> and
> >>>>>>>>>> 976 as blocker?
> >>>>>>>>>>
> >>>>>>>>>> Cheers
> >>>>>>>>>> Bolke
> >>>>>>>>>>
> >>>>>>>>>> Verstuurd vanaf mijn iPad
> >>>>>>>>>>
> >>>>>>>>>>> Op 28 sep. 2017 om 19:49 heeft Chris Riccomini
<
> >>>>>>> criccomini@apache.org <mailto:criccomini@apache.org>>
> >>>>>>>>>> het volgende geschreven:
> >>>>>>>>>>>
> >>>>>>>>>>> Hey all,
> >>>>>>>>>>>
> >>>>>>>>>>> I was planning to cut a 1.9.0 stable branch
and 1.9.0 beta
> >>>>>> release,
> >>>>>>> but
> >>>>>>>>>>> seeing as there are several outstanding
bugs, I'm going to
> >> delay.
> >>>>>>> Here
> >>>>>>>>>> are
> >>>>>>>>>>> the bugs that I'm tracking:
> >>>>>>>>>>>
> >>>>>>>>>>> AIRFLOW-1611 |Bug         |Customize logging
in Airflow
> >>>>>>>>>>> AIRFLOW-1525 |Improvement |Fix minor LICENSE
& NOTICE issue
> >>>>>>>>>>> AIRFLOW-1258 |Bug         |TaskInstances
within SubDagOperator
> >> are
> >>>>>>>>>> marked as
> >>>>>>>>>>> AIRFLOW-1055 |Bug         |airflow/jobs.py:create_dag_run()
> >>>>>>> exception
> >>>>>>>>>> for
> >>>>>>>>>>> @on
> >>>>>>>>>>> AIRFLOW-1018 |Bug         |Scheduler DAG
processes can not log
> >> to
> >>>>>>>>>> stdout
> >>>>>>>>>>> AIRFLOW-1013 |Bug         |airflow/jobs.py:manage_slas()
> >>>>>> exception
> >>>>>>>>>> for @once
> >>>>>>>>>>> AIRFLOW-988  |Bug         |SLA Miss Callbacks
Are Repeated if
> >>>>>> Email
> >>>>>>> is
> >>>>>>>>>> Not
> >>>>>>>>>>> be
> >>>>>>>>>>> AIRFLOW-976  |Bug         |Mark success
running task causes it
> >> to
> >>>>>>> fail
> >>>>>>>>>>>
> >>>>>>>>>>> These are the priority issues. Once they're
merged, I'll cut
> the
> >>>>>>>>>>> v1-9-stable and beta release.
> >>>>>>>>>>>
> >>>>>>>>>>> If you can help clean this up, that would
be really
> appreciated.
> >>>>>>>>>>>
> >>>>>>>>>>> Cheers,
> >>>>>>>>>>> Chris
> >>>>>>>>>>>
> >>>>>>>>>>> On Thu, Sep 28, 2017 at 10:06 AM, Chris
Riccomini <
> >>>>>>>>>> criccomini@apache.org <mailto:criccomini@apache.org>>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Marked it for 1.9.0.
> >>>>>>>>>>>>
> >>>>>>>>>>>>> On Thu, Sep 28, 2017 at 9:56 AM,
Charlie Jones <
> >>>>>> cjones@simpli.fi <mailto:cjones@simpli.fi>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Is there any chance we could include
AIRFLOW-988 in 1.9.0?
> SLA
> >>>>>>>>>> callbacks
> >>>>>>>>>>>>> are not working correctly without
emails... Its not a major
> >> bug,
> >>>>>>> but
> >>>>>>>>>> it
> >>>>>>>>>>>>> does cause us some annoyance in
our current deployment.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Link to Jira:
> >>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-988
<
> >> https://issues.apache.org/jira/browse/AIRFLOW-988>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Link to PR:
> >>>>>>>>>>>>> https://github.com/apache/incubator-airflow/pull/2415
<
> >> https://github.com/apache/incubator-airflow/pull/2415>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>> Charlie Jones
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> CHARLIE JONES
> >>>>>>>>>>>>> Data Engineer
> >>>>>>>>>>>>> cjones@simpli.fi <mailto:cjones@simpli.fi>
 |  M:
> >> 972.821.7631 <tel:972.821.7631>
> >>>>>>>>>>>>> __________________________________________________
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Programmatic Performance.* Localized.*
> >>>>>>>>>>>>> __________________________________________________
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 1407 Texas Street  |  Suite 202
 |  Fort Worth, TX 76102
> >>>>>>>>>>>>> 800.840.0768 <tel:800.840.0768>
 |  www.simpli.fi <
> >> http://www.simpli.fi/>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Wed, Sep 27, 2017 at 12:11 PM,
Chris Riccomini <
> >>>>>>>>>> criccomini@apache.org <mailto:criccomini@apache.org>>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Merged.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Tue, Sep 26, 2017 at 2:25
PM, Ryan Buckley <
> >>>>>>>>>>>>> ryan.buckley@bluecore.com <mailto:ryan.buckley@bluecore.com
> >>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Would it be possible to
include AIRFLOW-1587?
> >>>>>>>>>>>>>>> Running dags from the UI
is currently broken on the 1.9.0
> >>>>>> branch
> >>>>>>>>>> due
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>>> this issue.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> https://github.com/apache/incubator-airflow/pull/2590
<
> >> https://github.com/apache/incubator-airflow/pull/2590>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>> Ryan
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Tue, Sep 26, 2017 at
8:14 AM, Driesprong, Fokko
> >>>>>>>>>>>>> <fokko@driesprong.frl
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I would like to include
AIRFLOW-1611 in the 1.9.0 release:
> >>>>>>>>>>>>>>>> https://github.com/apache/incubator-airflow/pull/2631
<
> >> https://github.com/apache/incubator-airflow/pull/2631>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Currently importing
a custom logging configuration is not
> >>>>>> work
> >>>>>>> (as
> >>>>>>>>>>>>> far
> >>>>>>>>>>>>>>> as I
> >>>>>>>>>>>>>>>> know). Any feedback
on the PR would also be appreciated.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Cheers, Fokko
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> 2017-09-25 23:27 GMT+02:00
Chris Riccomini <
> >>>>>>> criccomini@apache.org <mailto:criccomini@apache.org>
> >>>>>>>>>>> :
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Done!
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Mon, Sep 25,
2017 at 5:11 AM, Michael Crawford <
> >>>>>>>>>>>>>>>>> michael.crawford@modernizingmedicine.com
<mailto:
> >> michael.crawford@modernizingmedicine.com>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Can you slide
the aws and emr connection type fix in?
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1636
<
> >> https://issues.apache.org/jira/browse/AIRFLOW-1636> <
> >>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1636
<
> >> https://issues.apache.org/jira/browse/AIRFLOW-1636>>
> >>>>>>>>>>>>>>>>>> https://github.com/apache/incubator-airflow/pull/2626
<
> >> https://github.com/apache/incubator-airflow/pull/2626> <
> >>>>>>>>>>>>>>>>>> https://github.com/apache/incubator-airflow/pull/2626
<
> >> https://github.com/apache/incubator-airflow/pull/2626>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> It keeps the
connection type from getting blanked out on
> >>>>>> edit
> >>>>>>>>>>>>> for
> >>>>>>>>>>>>>>> these
> >>>>>>>>>>>>>>>>>> types.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Sep 21,
2017, at 1:27 PM, Chris Riccomini <
> >>>>>>>>>>>>>>> criccomini@apache.org <mailto:criccomini@apache.org>>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Absolutely.
Just cherry-picked. I've been looking
> >> forward
> >>>>>> to
> >>>>>>>>>>>>>> these
> >>>>>>>>>>>>>>>>> fixes!
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Wed,
Sep 20, 2017 at 5:23 PM, Alex Guziel <
> >>>>>>>>>>>>>>> alex.guziel@airbnb.com <mailto:alex.guziel@airbnb.com>
> >>>>>>>>>>>>>>>> .
> >>>>>>>>>>>>>>>>>> invalid
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Can
we get this in?
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1519
<
> >> https://issues.apache.org/jira/browse/AIRFLOW-1519>
> >>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1621
<
> >> https://issues.apache.org/jira/browse/AIRFLOW-1621>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> https://github.com/apache/incubator-airflow/commit/
<
> >> https://github.com/apache/incubator-airflow/commit/>
> >>>>>>>>>>>>>>>>>>>> b6d2e0a46978e93e16576604624f57d1388814f2
> >>>>>>>>>>>>>>>>>>>> https://github.com/apache/incubator-airflow/commit/
<
> >> https://github.com/apache/incubator-airflow/commit/>
> >>>>>>>>>>>>>>>>>>>> 656d045e90bf67ca484a3778b2a07a419bfb324a
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> It speeds
up loading times a lot, so it's a good thing
> >> to
> >>>>>>>>>>>>> have
> >>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>> 1.9.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Wed,
Sep 20, 2017 at 11:14 AM, Chris Riccomini <
> >>>>>>>>>>>>>>>>>> criccomini@apache.org
<mailto:criccomini@apache.org>>
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
Sounds good. I'll plan on stable+beta next week,
> then.
> >>>>>>>>>>>>> Initial
> >>>>>>>>>>>>>>>>> warning
> >>>>>>>>>>>>>>>>>>>>>
stands, that I will start locking down what can get
> >> into
> >>>>>>>>>>>>> 1.9.0
> >>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>
point.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
On Wed, Sep 20, 2017 at 11:10 AM, Bolke de Bruin <
> >>>>>>>>>>>>>>>> bdbruin@gmail.com <mailto:bdbruin@gmail.com>>
> >>>>>>>>>>>>>>>>>>>>>
wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
No vote indeed, just to gather feedback on a
> >> particular
> >>>>>>>>>>>>> fixed
> >>>>>>>>>>>>>>>> point
> >>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>
time. It also gives a bit more trust to a tarball
> >> than
> >>>>>> to
> >>>>>>> a
> >>>>>>>>>>>>>> git
> >>>>>>>>>>>>>>>>> pull.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
Bolke
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
On 20 Sep 2017, at 20:09, Chris Riccomini <
> >>>>>>>>>>>>>>> criccomini@apache.org <mailto:criccomini@apache.org>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
I can do a beta. Is the process significantly
> >>>>>> different?
> >>>>>>>>>>>>>> IIRC,
> >>>>>>>>>>>>>>>> it's
> >>>>>>>>>>>>>>>>>>>>>>>
basically the same, just no vote, right?
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
On Wed, Sep 20, 2017 at 10:56 AM, Bolke de Bruin <
> >>>>>>>>>>>>>>>>> bdbruin@gmail.com
<mailto:bdbruin@gmail.com>>
> >>>>>>>>>>>>>>>>>>>>>>
wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
Are you sure you want to go ahead and do RCs right
> >>>>>> away?
> >>>>>>>>>>>>>>> Isn’t a
> >>>>>>>>>>>>>>>>>>>> beta
> >>>>>>>>>>>>>>>>>>>>>
a
> >>>>>>>>>>>>>>>>>>>>>>>>
bit smarter?
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
- Bolke
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
On 20 Sep 2017, at 19:41, Chris Riccomini <
> >>>>>>>>>>>>>>>> criccomini@apache.org
<mailto:criccomini@apache.org>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
Hey all,
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
I want to send out a warning that I'm planning to
> >>>>>> cut
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>> stable
> >>>>>>>>>>>>>>>>>>>>>
branch
> >>>>>>>>>>>>>>>>>>>>>>>>>
next week, and begin the RC1 release vote. Once
> >> the
> >>>>>>>>>>>>> stable
> >>>>>>>>>>>>>>>> branch
> >>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>
cut, I
> >>>>>>>>>>>>>>>>>>>>>>>>>
will be locking down what commits get cherry
> >> picked
> >>>>>>> into
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> branch,
> >>>>>>>>>>>>>>>>>>>>>>
and
> >>>>>>>>>>>>>>>>>>>>>>>>>
will only be doing PRs that are required to get
> >> the
> >>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>> out.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
Cheers,
> >>>>>>>>>>>>>>>>>>>>>>>>>
Chris
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
On Mon, Sep 18, 2017 at 11:19 AM, Chris Riccomini
> >> <
> >>>>>>>>>>>>>>>>>>>>>>
criccomini@apache.org <mailto:criccomini@apache.org
> >
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
Hey all,
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
An update on the 1.9.0 release. Here are the
> >>>>>>>>>>>>> outstanding
> >>>>>>>>>>>>>> PRs
> >>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>
are
> >>>>>>>>>>>>>>>>>>>>>>>>>>
slated to be included into 1.9.0:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
ISSUE ID     |STATUS    |DESCRIPTION
> >>>>>>>>>>>>>>>>>>>>>>>>>>
AIRFLOW-1617 |Open      |XSS Vulnerability in
> >>>>>> Variable
> >>>>>>>>>>>>>>>> endpoint
> >>>>>>>>>>>>>>>>>>>>>>>>>>
AIRFLOW-1611 |Open      |Customize logging in
> >>>>>> Airflow
> >>>>>>>>>>>>>>>>>>>>>>>>>>
AIRFLOW-1605 |Reopened  |Fix log source of local
> >>>>>>>>>>>>> loggers
> >>>>>>>>>>>>>>>>>>>>>>>>>>
AIRFLOW-1604 |Open      |Rename the logger to
> log
> >>>>>>>>>>>>>>>>>>>>>>>>>>
AIRFLOW-1525 |Open      |Fix minor LICENSE &
> >> NOTICE
> >>>>>>>>>>>>> issue
> >>>>>>>>>>>>>>>>>>>>>>>>>>
AIRFLOW-1499 |In Progres|Eliminate duplicate and
> >>>>>>>>>>>>> unneeded
> >>>>>>>>>>>>>>> code
> >>>>>>>>>>>>>>>>>>>>>>>>>>
AIRFLOW-1198 |Open      |HDFSOperator to operate
> >>>>>> HDFS
> >>>>>>>>>>>>>>>>>>>>>>>>>>
AIRFLOW-1055 |Open
> >>>>>> |airflow/jobs.py:create_dag_ru
> >>>>>>>>>>>>> n()
> >>>>>>>>>>>>>>>>>>>> exception
> >>>>>>>>>>>>>>>>>>>>>>
for
> >>>>>>>>>>>>>>>>>>>>>>>>>>
@on
> >>>>>>>>>>>>>>>>>>>>>>>>>>
AIRFLOW-1019 |Open      |active_dagruns
> shouldn't
> >>>>>>>>>>>>> include
> >>>>>>>>>>>>>>>> paused
> >>>>>>>>>>>>>>>>>>>>>
DAGs
> >>>>>>>>>>>>>>>>>>>>>>>>>>
AIRFLOW-1018 |Open      |Scheduler DAG processes
> >>>>>> can
> >>>>>>>>>>>>> not
> >>>>>>>>>>>>>> log
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>
stdout
> >>>>>>>>>>>>>>>>>>>>>>>>>>
AIRFLOW-1015 |Open      |TreeView displayed over
> >>>>>> task
> >>>>>>>>>>>>>>>> instances
> >>>>>>>>>>>>>>>>>>>>>>>>>>
AIRFLOW-1013 |Open
> >>>>>> |airflow/jobs.py:manage_slas()
> >>>>>>>>>>>>>>>> exception
> >>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>
@once
> >>>>>>>>>>>>>>>>>>>>>>>>>>
AIRFLOW-976  |Open      |Mark success running
> >> task
> >>>>>>>>>>>>> causes
> >>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>> fail
> >>>>>>>>>>>>>>>>>>>>>>>>>>
AIRFLOW-914  |Open      |Refactor
> >>>>>>>>>>>>>>>> BackfillJobTest.test_backfill_
> >>>>>>>>>>>>>>>>>>>>>>>>
examples
> >>>>>>>>>>>>>>>>>>>>>>>>>>
to
> >>>>>>>>>>>>>>>>>>>>>>>>>>
AIRFLOW-913  |Open      |Refactor
> >>>>>>>>>>>>>>>> tests.CoreTest.test_scheduler_
> >>>>>>>>>>>>>>>>>>>> job
> >>>>>>>>>>>>>>>>>>>>>>
to
> >>>>>>>>>>>>>>>>>>>>>>>>>>
real
> >>>>>>>>>>>>>>>>>>>>>>>>>>
AIRFLOW-912  |Open      |Refactor tests and
> build
> >>>>>>>>>>>>> matrix
> >>>>>>>>>>>>>>>>>>>>>>>>>>
AIRFLOW-888  |Open      |Operators should not
> >> push
> >>>>>>>>>>>>> XComs
> >>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>> default
> >>>>>>>>>>>>>>>>>>>>>>>>>>
AIRFLOW-828  |Open      |Add maximum size for
> >> XComs
> >>>>>>>>>>>>>>>>>>>>>>>>>>
AIRFLOW-825  |Open      |Add Dataflow semantics
> >>>>>>>>>>>>>>>>>>>>>>>>>>
AIRFLOW-788  |Open      |Context unexpectedly
> >>>>>> added to
> >>>>>>>>>>>>>> hive
> >>>>>>>>>>>>>>>> conf
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
I will be locking down what can get
> cherry-picked
> >>>>>> into
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>> 1.9.0
> >>>>>>>>>>>>>>>>>>>>>>
branch
> >>>>>>>>>>>>>>>>>>>>>>>>>>
shortly, so if you have something you want in,
> >>>>>> please
> >>>>>>>>>>>>> set
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> fix
> >>>>>>>>>>>>>>>>>>>>>>>>
version
> >>>>>>>>>>>>>>>>>>>>>>>>>>
to 1.9.0.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
We (at WePay) have deployed 1.9.0 into our dev
> >>>>>>> cluster,
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>> has
> >>>>>>>>>>>>>>>>>>>>>>
been
> >>>>>>>>>>>>>>>>>>>>>>>>>>
running smoothly for several days.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
** I could really use help verifying stability.
> >> If
> >>>>>> you
> >>>>>>>>>>>>> run
> >>>>>>>>>>>>>>>>>>>> Airflow,
> >>>>>>>>>>>>>>>>>>>>>>
it's
> >>>>>>>>>>>>>>>>>>>>>>>>>>
in your best interest to deploy the 1.9.0 test
> >>>>>> branch
> >>>>>>>>>>>>>>>> somewhere,
> >>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>
verify
> >>>>>>>>>>>>>>>>>>>>>>>>>>
it's working for your workload. **
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
Cheers,
> >>>>>>>>>>>>>>>>>>>>>>>>>>
Chris
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>
> >>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message