airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siddharth Anand (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-48) can't pass extra parameters via a connection URI when using environment variables (and therefore can't tell Postgres to use SSL)
Date Fri, 06 May 2016 00:41:12 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-48?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15273398#comment-15273398
] 

Siddharth Anand commented on AIRFLOW-48:
----------------------------------------

It's likely not implemented because no one is using it. Can you provide an example of your
uri?

Is it something like *postgresql://localhost:2345/your_db?sslmode=require*?

What version of postgresql are you running? I'm not sure how well this parameter is supported
for versions of postgresql before 9.2.

Another alternative is to take control of this by implementing this logic in a PythonOperator's
python callable. That could read this param and use psycopg2 to connect to your db. Of course,
then you would not be using the Connection or Hook, but, you would be able to achieve your
end goal. 

In the meantime, feel free to submit a PR for review -- we'd be happy to accept it.  

> can't pass extra parameters via a connection URI when using environment variables (and
therefore can't tell Postgres to use SSL)
> --------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: AIRFLOW-48
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-48
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: hooks
>    Affects Versions: Airflow 1.7.0
>         Environment: Ubuntu 14.04.3
>            Reporter: Matt Prast
>
> We're trying to use a postgres connection to hook into our Redshift backend (which has
sslmode set to 'require'). Since we're using chef to deploy we've got the connection string
stored in an environment variable, but it looks like the Connection model doesn't parse any
connection parameters when it gets initialized with a uri (https://github.com/airbnb/airflow/blob/3c3f5a67ff80f3e8942aef441f481c62baf97184/airflow/models.py#L529).
We can't find a way to tell postgres to use SSL without setting sslmode via the connection
string, but airflow doesn't seem to allow this, even though you can do it through the UI.
Is there something we're missing? Has anybody else run into this using Chef + Postgres + SSL?
Thanks for your help!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message