airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "George Leslie-Waksman (JIRA)" <>
Subject [jira] [Created] (AIRFLOW-1491) Celery executor restarts result in duplicate tasks
Date Mon, 07 Aug 2017 18:06:00 GMT
George Leslie-Waksman created AIRFLOW-1491:

             Summary: Celery executor restarts result in duplicate tasks
                 Key: AIRFLOW-1491
             Project: Apache Airflow
          Issue Type: Bug
            Reporter: George Leslie-Waksman
            Assignee: George Leslie-Waksman

Since the Celery executor uses internal state to determine what it has queued already, anything
that wipes the executor state (such as a scheduler restart) will result in rescheduling of

This bug exacerbates a number of task scheduling race conditions and can result in duplicate
task execution.

This message was sent by Atlassian JIRA

View raw message