airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gabriele Di Bernardo <gdiberna...@bol.com>
Subject Sync DAGs folder with Git
Date Thu, 11 Apr 2019 15:44:30 GMT
Hello everyone,

I am relatively new to Airflow and I am not sure if this is the right place to ask questions.

We set up a simple Kubernetes pod running a Docker image that contains Airflow. We want the
DAGs to be synced from a Git repository. The first idea we had was to define an extra dag
responsible for syncing the dag folder with the remote repository. However, I saw that in
the airflow.cfg file there are some entries related to Git:

# Git credentials and repository for DAGs mounted via Git (mutually exclusive with volume
claim)
git_repo =
git_branch =
git_subpath =
# Use git_user and git_password for user authentication or git_ssh_key_secret_name and git_ssh_key_secret_key
# for SSH authentication
git_user =
git_password =
git_sync_root = /git
git_sync_dest = repo
# Mount point of the volume if git-sync is being used.
# i.e. {AIRFLOW_HOME}/dags
git_dags_folder_mount_point =

So I was wondering what is the best way to keep in sync Airflow with a remote repository that
contains DAGs and if this functionality is already integrated natively in Airflow!

Thank you so much!

Best,


Gabriele

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message