airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maxime Beauchemin <>
Subject Re: AIP-8 Split Hooks/Operators into Separate Packages
Date Mon, 07 Jan 2019 20:16:55 GMT
Something to think about is how data transfer operators like the
MysqlToHiveOperator usually rely on 2 hooks. With a package-specific
approach that may mean something like an `airflow-hive`, `airflow-mysql`
and `airflow-mysql-hive` packages, where the `airflow-mysql-hive` package
depends on the two other packages.

It's just a matter of having a clear strategy, good naming conventions and
a nice central place in the docs that centralize a list of approved


On Mon, Jan 7, 2019 at 9:05 AM Tim Swast <> wrote:

> I've created AIP-8:
> To follow-up from the discussion about splitting hooks/operators out of the
> core Airflow package at
> I propose packaging based on the target system, informed by the existing
> hooks in both core and contrib. This will allow those with the relevant
> expertise in each target system to respond to contributions / issues
> without having to follow the flood of everything Airflow-related. It will
> also decrease the surface area of the core package, helping with
> testability and long-term maintenance.
> *  •  **Tim Swast*
> *  •  *Software Friendliness Engineer
> *  •  *Google Cloud Developer Relations
> *  •  *Seattle, WA, USA

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message