airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bolke de Bruin (JIRA)" <j...@apache.org>
Subject [jira] [Created] (AIRFLOW-791) At start up all running dag_runs are being checked, but not fixed
Date Sun, 22 Jan 2017 12:21:26 GMT
Bolke de Bruin created AIRFLOW-791:
--------------------------------------

             Summary: At start up all running dag_runs are being checked, but not fixed
                 Key: AIRFLOW-791
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-791
             Project: Apache Airflow
          Issue Type: Bug
            Reporter: Bolke de Bruin


At start up all running dag_runs are being checked, we seemed to have a lot of “left over”
dag_runs (couple of thousand)
- Checking was logged to INFO -> requires a fsync for every log message making it very
slow
- Checking would happen at every restart, but dag_runs’ states were not being updated
- These dag_runs would never er be marked anything else than running for some reason
-> Applied work around to update all dag_run in sql before a certain date to -> finished
-> need to investigate why dag_runs did not get marked “finished/failed” 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message