airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Standish (Jira)" <j...@apache.org>
Subject [jira] [Updated] (AIRFLOW-5768) Google Cloud SQL - Don't store ephemeral connection object to database
Date Sun, 27 Oct 2019 04:31:00 GMT

     [ https://issues.apache.org/jira/browse/AIRFLOW-5768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Daniel Standish updated AIRFLOW-5768:
-------------------------------------
    Description: 
GCP cloud sql operator creates dynamically an ephemeral Connection object.  It persists to
metastore during execution and deletes afterward.  

This behavior has negative impact on our ability to refactor creds management.  

By not persisting to database, we can also remove some complexity re ensuring connection is
deleted in event of failure, and the tests that go along with that.

It does require that we add optional param `connection` to both MySqlHook and PostgresHook.


  was:
GCP cloud sql operator creates dynamically an ephemeral Connection object.  It persists to
metastore during execution and deletes afterward.  

This behavior has negative impact on our ability to refactor creds management.  

By not persisting to database, we can also remove some complexity re ensuring connection is
deleted in event of failure, and the tests that go along with that.


> Google Cloud SQL - Don't store ephemeral connection object to database
> ----------------------------------------------------------------------
>
>                 Key: AIRFLOW-5768
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-5768
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: gcp
>    Affects Versions: 1.10.5
>            Reporter: Daniel Standish
>            Assignee: Daniel Standish
>            Priority: Major
>
> GCP cloud sql operator creates dynamically an ephemeral Connection object.  It persists
to metastore during execution and deletes afterward.  
> This behavior has negative impact on our ability to refactor creds management.  
> By not persisting to database, we can also remove some complexity re ensuring connection
is deleted in event of failure, and the tests that go along with that.
> It does require that we add optional param `connection` to both MySqlHook and PostgresHook.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message