airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Grant Nicholas (JIRA)" <j...@apache.org>
Subject [jira] [Created] (AIRFLOW-1884) Ensure scheduler is crash safe for externally triggered and backfilled dagruns
Date Mon, 04 Dec 2017 19:40:00 GMT
Grant Nicholas created AIRFLOW-1884:
---------------------------------------

             Summary: Ensure scheduler is crash safe for externally triggered and backfilled
dagruns
                 Key: AIRFLOW-1884
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1884
             Project: Apache Airflow
          Issue Type: Bug
            Reporter: Grant Nicholas
            Assignee: Grant Nicholas


Orphaned task instances are only reset for dagruns that are both not externally triggered
and not backfilled. This violates the crash safety property of the scheduler, ie) if the scheduler
crashes in the middle of one of these dagruns then tasks can be stuck in the "Queued" state
forever and never executed. 

I found the changeset this regression happened in, it is this one:
https://issues.apache.org/jira/browse/AIRFLOW-1059

This change reverts the special casing logic so that all dagruns have orphaned tasks reset
on startup of the scheduler. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message