airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Isaac Steele (JIRA)" <j...@apache.org>
Subject [jira] [Created] (AIRFLOW-323) Should be able to prevent tasks from overlapping across multiple DAG Runs
Date Mon, 11 Jul 2016 16:07:11 GMT
Isaac Steele created AIRFLOW-323:
------------------------------------

             Summary: Should be able to prevent tasks from overlapping across multiple DAG
Runs
                 Key: AIRFLOW-323
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-323
             Project: Apache Airflow
          Issue Type: Bug
    Affects Versions: Airflow 1.7.1.2
         Environment: 1.7.1.2
            Reporter: Isaac Steele
            Assignee: Isaac Steele


As a the Airflow administrator,
If a task from a previous DAG Run is still running when the next scheduled run triggers the
same task, there should be a way prevent the tasks from overlapping.
Otherwise the same code could end up running multiple times simultaneously.

To reproduce:
1) Create a DAG with a short scheduled interval
2) Create a task in that DAG to run longer than the interval
Result: Both tasks end up running that the same time.

This can cause tasks to compete for resources as well as duplicating or overwriting what the
other task is doing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message