airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Barry Hart (JIRA)" <>
Subject [jira] [Commented] (AIRFLOW-15) Remove GCloud from Airflow
Date Tue, 05 Dec 2017 21:15:00 GMT


Barry Hart commented on AIRFLOW-15:

Question: This change moved Airflow to using a library that Google no longer recommends (
If you're working with Google Cloud Platform APIs such as Datastore or Pub/Sub, consider using
the Cloud Client Libraries for Python instead. These are the new and idiomatic Python libraries
targeted specifically at Google Cloud Platform Services.

Should this decision be revisited and possibly reversed? I am happy to open a new ticket,
but wanted to raise the question here first for context.

> Remove GCloud from Airflow
> --------------------------
>                 Key: AIRFLOW-15
>                 URL:
>             Project: Apache Airflow
>          Issue Type: Task
>          Components: gcp
>            Reporter: Chris Riccomini
>            Assignee: Chris Riccomini
>              Labels: gcp
> After speaking with Google, there was some concern about using the [gcloud-python|]
library for Airflow. There are several concerns:
> # It's not clear (even to people at Google) what this library is, who owns it, etc.
> # It does not support all services (the way [google-api-python-client|]
> # There are compatibility issues between google-api-python-client and gcloudpython.
> We currently support both, after libraries depending on which package you you install:
{{airfow[gcp_api]}} or {{airflow[gcloud]}}. This ticket is to remove the {{airflow[gcloud]}}
packaged, and all associated code.
> The main associated code, afaik, is the use of the {{gcloud}} library in the Google cloud
storage hooks/operators--specifically for Google cloud storage Airfow logging.

This message was sent by Atlassian JIRA

View raw message