airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yannick Einsweiler (JIRA)" <>
Subject [jira] [Commented] (AIRFLOW-1894) Rebase and migrate existing Airflow GCP operators to google-python-cloud
Date Tue, 17 Apr 2018 21:13:00 GMT


Yannick Einsweiler commented on AIRFLOW-1894:

[~wileeam] read my initial comment, the idea was to leverage []
*without* the need for a local or virtual machine passthrough.

We ended writing that operator (and a sensor that checks when job is complete), all triggered
from within Gcloud but have since then abandoned the project and have our AWS Lambda save
output to GCS directly.

> Rebase and migrate existing Airflow GCP operators to google-python-cloud
> ------------------------------------------------------------------------
>                 Key: AIRFLOW-1894
>                 URL:
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: contrib
>    Affects Versions: Airflow 2.0
>            Reporter: Feng Lu
>            Assignee: Feng Lu
>            Priority: Minor
> [google-api-python-client|] is in maintenance
mode and it's recommended that [google-cloud-python|]
should be used whenever possible. Given that we don't have feature parity between the two
libraries, this issue is created to track the long-term migration efforts moving from google-api-python-client
to google-cloud-python. Here are some general guidelines we try to follow in this cleanup
> - add google-cloud-python dependency as part of gcp_api extra packages (make sure there
is no dependency conflict between the two).
> - new operators shall be based on google-cloud-python if possible.
> - migrate existing GCP operators when the underlying GCP service is available in google-cloud-python.

This message was sent by Atlassian JIRA

View raw message