airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Song Liu <>
Subject About the DAG discovering not synced between scheduler and webserver
Date Sat, 12 May 2018 11:58:43 GMT

When add a new dag, sometimes we can see:

This DAG isn't available in the web server's DagBag object. It shows up in this list because
the scheduler marked it as active in the metadata database.

In the, it will collect DAGs under "DAGS_FOLDER" by instantiate a DagBag object as

dagbag = models.DagBag(settings.DAGS_FOLDER)

So that webserver will depends on its own timing to collect DAGs, but why not just simply
to query metadata db ? since if a DAG is active in DB now it can be visible in web at the

Could someone share something behind this design ?

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message