airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jarek Potiuk <Jarek.Pot...@polidea.com>
Subject Re: [Proposal] Component changes for Apache Airflow
Date Thu, 30 May 2019 03:52:08 GMT
I think there is a discussion (and some work already done) to move more
mature operators/hooks from the 'contrib' directory to the main directory
for hooks/operators so I think "contrib-" prefix might be too easily
confused with the *"*contrib*"* folder until this is finalised.

I think we need a better "abstraction" name for "operators+hooks" as we
might also use it in the upcoming documentation update. Something that will
easily separate out "core" from the "operators+hooks" specific for
particular external system.

I am afraid I will open Pandora's box :) (we all know naming is hard
<https://hilton.org.uk/blog/why-naming-things-is-hard>), but  Maybe "ext"
might be a good prefix ("ext-gcp", "ext-kubernetes", "ext-aws"). It can be
understood as both "extension" and "external" depending whether it is used
in context of Airflow (extension) or the Operator (external system to
connect to)?

J.

On Wed, May 29, 2019 at 11:50 PM Aizhamal Nurmamat kyzy
<aizhamal@google.com.invalid> wrote:

> Hi Daniel,
>
> The reason I had removed `kubernetes` component is that the component
> attracted two different kinds of issues. I would like to be able to
> differentiate issues related to kubernetes executor, and kubernetes-related
> operators and hooks. I have been thinking that prefixes may be helpful in
> this case.
>
> - For operators/hooks, we can have a: `contrib-kubernetes` component.
> - For the Kubernetes executor, we can have a `executor-kubernetes`
> component.
>
> I think this prefix structure would be really helpful, as it sets a
> precedent for other components for executors (e.g. `executor-celery`,
> `executors` (for any without a particular component)); and for
> operators/hook (e.g. `contrib-gcp`, `contrib-aws`, ...). This allows
> users/contributors to file issues, and see the appropriate components as
> they type the prefix.
>
> Thoughts?
>


-- 

Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer

M: +48 660 796 129 <+48660796129>
E: jarek.potiuk@polidea.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message