airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Guziel (JIRA)" <>
Subject [jira] [Created] (AIRFLOW-1493) Fix race condition with airflow run
Date Tue, 08 Aug 2017 01:49:00 GMT
Alex Guziel created AIRFLOW-1493:

             Summary: Fix race condition with airflow run
                 Key: AIRFLOW-1493
             Project: Apache Airflow
          Issue Type: Bug
            Reporter: Alex Guziel
            Assignee: Alex Guziel

Currently, airflow run spawns a process `airflow run --local` which spawns `airflow run --raw`.

Local manages the heartbeat. Raw performs a series of checks, sets the state to running, runs
the task, then sets the state to failed or success. 

The problem is the heartbeat check on `airflow run --local` has to monitor the state in the
DB, but because the change of state to running happens asynchronously, it must first observe
the state in the DB to be running before it has the power of termination. However, there is
no guarantee that it will observe this state. Thus, we should  move the pre-execution logic
to airflow run --local

This message was sent by Atlassian JIRA

View raw message