airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zsolt Tóth <toth.zsolt....@gmail.com>
Subject Re: Airflow webserver forgets dags on refresh
Date Wed, 19 Jul 2017 11:56:34 GMT
Follow-up: I found that this issue happens when the DAG file uses a common
utility method to create the Dag object, by passing some parameters to it.
By looking into the airflow DB, the fileloc column contains the path to the
utility file (e.g. $DAG_FOLDER/utils/dagfactory.py) instead of the path of
the dag itself (which would be $DAG_FOLDER/mydag.py). In 1.7.x, the
location is correct. I suspect that this may be related to the issue.
Any ideas on this?

2017-07-18 18:25 GMT+02:00 Zsolt Tóth <toth.zsolt.bme@gmail.com>:

> Hi,
>
> I'm using Airflow 1.8.1, but also tried with 1.8.0 and 1.8.2-rc1, with
> LocalExecutor, on a single node. When I click Refresh on the UI (either in
> the Dag's page, or in the main DAG list), there is a good chance that
> airflow somehow does not find the DAG anymore and displays the stacktrace
> with the usual error message, when it can't find a dag:
>
> AttributeError: 'NoneType' object has no attribute 'latest_execution_date'
>
> After some time (usually ~15s) if I refresh the browser, the DAG is found
> and everything is back to normal.
> Checked the webserver and scheduler logs, but couldn't find anything else
> than the stacktrace that's displayed on the UI.
> This happens frequently, usually 6-7 out of 10 refreshes. Any idea what
> goes wrong here? The permissions should be fine, and the postgres DB seems
> to contain all the dags.
>
>
> Thanks,
> Zsolt
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message