airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Imberman <daniel.imber...@gmail.com>
Subject AIP-25 The Knative Executor
Date Fri, 16 Aug 2019 15:17:38 GMT
Hello all!

Over the last two years, there has been a lot of incredible adoption and
feedback regarding the KubernetesExecutor. While there have been many
positive improvements within this feature, one thing that has been commonly
requested is a better story around scaling and start-up time. I (and some
extremely helpful colleagues) wanted to come up with a solution that could
both offer the dynamic allocation of the KubernetesExecutor with the
scale/start-up time improvements of having shared airflow machines. With
these constraints in mind, I want to propose a Knative Executor.

I've written up an AIP for the wiki below

https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-25+The+Knative+Executor

But for early edits I would like to offer this google doc link for early
edits (the google doc will allow for in-line comments and edit suggestions).

https://docs.google.com/document/d/1xcSjkYv9Sh1Vavb-tvLUdy-8AHetrWKfiIblQE1rdYc/edit?usp=sharing

I'm really excited to get feedback on this and hope that this is the part
of an exciting new chapter in the airflow + kubernetes integration :).

Thank you all for your consideration on this AIP!

Daniel

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message