airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "peay (JIRA)" <>
Subject [jira] [Created] (AIRFLOW-1431) Cannot create connection for GCP using CLI
Date Wed, 19 Jul 2017 13:52:00 GMT
peay created AIRFLOW-1431:

             Summary: Cannot create connection for GCP using CLI
                 Key: AIRFLOW-1431
             Project: Apache Airflow
          Issue Type: Bug
          Components: cli
    Affects Versions: Airflow 1.8
         Environment: Python 3
            Reporter: peay

{{airflow connections --add}} only takes an URI argument, and deduces the connection type
from the scheme, and other fields from hostname, etc.

The connection type for the GCP connection is {{google_cloud_platform}}.

This is not a valid scheme according to {{urllib.parse.urlparse}}:

>>> from urllib.parse import urlparse
>>> urlparse("google_cloud_platform://hostname")
ParseResult(scheme='', netloc='', path='google_cloud_platform://hostname', params='', query='',
>>> urlparse("platform://hostname")
ParseResult(scheme='platform', netloc='hostname', path='', params='', query='', fragment='')

See which specifies {{scheme      = ALPHA
*( ALPHA / DIGIT / "+" / "-" / "." )}}

 As a consequence, it is not currently possible to add GCP connections using the CLI.

Changing the connection name to {{gcp}} would solve it properly but may require lots of small
modifications accross the codebase. Alternatively, 

if scheme == "gcp":
    schema = "google_cloud_platform"

right after after parsing should be a simple self-contained fix. There is already a similar
fix in there for {{postgres -> postgresql}}. On the downside, this introduces a special
case that would need to be documented.

A last option would be to add an argument to override the scheme from the URI. This would
be backward compatible.

I'll be happy to contribute a PR if we can agree on a plan.

This message was sent by Atlassian JIRA

View raw message