airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <>
Subject [jira] [Commented] (AIRFLOW-2839) Answer a question in doc: what if I have connections with the same conn_id in both metadata and environment variables?
Date Sun, 05 Aug 2018 18:09:00 GMT


ASF GitHub Bot commented on AIRFLOW-2839:

kaxil closed pull request #3678: [AIRFLOW-2839] Refine Doc Concepts->Connections

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/docs/concepts.rst b/docs/concepts.rst
index f879ced012..09c1293806 100644
--- a/docs/concepts.rst
+++ b/docs/concepts.rst
@@ -333,6 +333,17 @@ from ``BaseHook``, Airflow will choose one connection randomly, allowing
 for some basic load balancing and fault tolerance when used in conjunction
 with retries.
+Airflow also has the ability to reference connections via environment
+variables from the operating system. But it only supports URI format. If you
+need to specify ``extra`` for your connection, please use web UI.
+If connections with the same ``conn_id`` are defined in both Airflow metadata
+database and environment variables, only the one in environment variables
+will be referenced by Airflow (for example, given ``conn_id`` ``postgres_master``,
+Airflow will search for ``AIRFLOW_CONN_POSTGRES_MASTER``
+in environment variables first and directly reference it if found,
+before it starts to search in metadata database).
 Many hooks have a default ``conn_id``, where operators using that hook do not
 need to supply an explicit connection ID. For example, the default
 ``conn_id`` for the :class:`~airflow.hooks.postgres_hook.PostgresHook` is


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:

> Answer a question in doc: what if I have connections with the same conn_id in both metadata
and environment variables?
> ----------------------------------------------------------------------------------------------------------------------
>                 Key: AIRFLOW-2839
>                 URL:
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: configuration, Documentation
>            Reporter: Xiaodong DENG
>            Assignee: Xiaodong DENG
>            Priority: Critical
>             Fix For: 2.0.0
> We can define connections either in metadata database or environment variables.
> However, how about there are connections with the same conn_id in both metadata database
and environment variables? Will one of them be picked? Which one? Or a random one will be
> The answer is: Airflow will check environment variables first.[] 
The "random picking" mechanism will only be applicable for multiple connections with the same
conn_id in metadata database (if there is no connection with the same conn_id defined in environment
> This should be made clear in the documentation.

This message was sent by Atlassian JIRA

View raw message