airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Feng Lu <fen...@google.com.INVALID>
Subject Adding airflow top-level label(s)
Date Tue, 06 Jun 2017 22:44:20 GMT
Hi,

Resource label/tag has gained increasing popularity, for example, both GCP
and AWS allow labels/tags to be attached to the cloud resources they
offered. I am wondering whether the airflow community is interested in
adding some default top-level airflow labels for all cloud resources that
support labeling, spawned from airflow. In other words, all operators
creating/using these resources will include the airflow labels (e.g.,
"airflow-version" : "1.8.1").

The benefits are twofold:
- enable measuring the impact of apache airflow quantitatively at different
places.
- allow users to better organize airflow triggered cloud resources.

These top-level labels will then be merged with user-provided labels
specified in the various operators (e.g.
https://github.com/apache/incubator-airflow/blob/master/airflow/contrib/operators/dataproc_operator.py#L55
).

Thanks.

Feng

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message