airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Noël BARDELOT (Jira) <j...@apache.org>
Subject [jira] [Updated] (AIRFLOW-6260) Drive _cmd config options by env var
Date Mon, 16 Dec 2019 15:59:00 GMT

     [ https://issues.apache.org/jira/browse/AIRFLOW-6260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Noël BARDELOT updated AIRFLOW-6260:
-----------------------------------
    Description: 
Add the ability to provide _CMD environment variables the same way you can use _cmd options
in the configuration. Those _CMD environment variable would have priority against the _cmd
options the same way the usual environment variables have priority against the configuration
options.

The goal is to allow Kubernetes users to provide such commands via configmap, and reference
them in the configuration using environment variables. The referenced script would itself
consume secrets in order to construct the connection.

For exemple a script provided as a configmap and referenced by AIRFLOW_CORE_SQL_ALCHEMY_CONN_CMD
would compute the Alchemy connection using the database user and password provided as secrets:

{{ }}

{{#!/bin/bash}}{{echo -n "postgresql+psycopg2://`cat /var/run/secrets/postgresql/postgresql-user`:`cat
/var/run/secrets/postgresql/postgresql-password`@$POSTGRESQL_SERVICE_HOST:$POSTGRESQL_SERVICE_PORT/airflow"}}

  was:
Add the ability to provide _CMD environment variables the same way you can use _cmd options
in the configuration. Those _CMD environment variable would have priority against the _cmd
options the same way the usual environment variables have priority against the configuration
options.

The goal is to allow Kubernetes users to provide such commands via configmap, and reference
them in the configuration using environment variables. The referenced script would itself
consume secrets in order to construct the connection.

For exemple a script provided as a configmap and referenced by AIRFLOW_CORE_SQL_ALCHEMY_CONN_CMD
would compute the Alchemy connection using the database user and password provided as secrets:

{{ }}
 \{{     #!/bin/bash}}
 \{{     echo -n "postgresql+psycopg2://`cat /var/run/secrets/postgresql/postgresql-user`:`cat
/var/run/secrets/postgresql/postgresql-password`@$POSTGRESQL_SERVICE_HOST:$POSTGRESQL_SERVICE_PORT/airflow"}}


> Drive _cmd config options by env var 
> -------------------------------------
>
>                 Key: AIRFLOW-6260
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-6260
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: configuration
>    Affects Versions: 1.10.6
>            Reporter: Noël BARDELOT
>            Assignee: Noël BARDELOT
>            Priority: Minor
>
> Add the ability to provide _CMD environment variables the same way you can use _cmd options
in the configuration. Those _CMD environment variable would have priority against the _cmd
options the same way the usual environment variables have priority against the configuration
options.
> The goal is to allow Kubernetes users to provide such commands via configmap, and reference
them in the configuration using environment variables. The referenced script would itself
consume secrets in order to construct the connection.
> For exemple a script provided as a configmap and referenced by AIRFLOW_CORE_SQL_ALCHEMY_CONN_CMD
would compute the Alchemy connection using the database user and password provided as secrets:
> {{ }}
> {{#!/bin/bash}}{{echo -n "postgresql+psycopg2://`cat /var/run/secrets/postgresql/postgresql-user`:`cat
/var/run/secrets/postgresql/postgresql-password`@$POSTGRESQL_SERVICE_HOST:$POSTGRESQL_SERVICE_PORT/airflow"}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message