airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bryan Vanderhoof (JIRA)" <j...@apache.org>
Subject [jira] [Created] (AIRFLOW-1204) Scheduler ignores start_date if an earlier successful dag run exists
Date Mon, 15 May 2017 20:39:05 GMT
Bryan Vanderhoof created AIRFLOW-1204:
-----------------------------------------

             Summary: Scheduler ignores start_date if an earlier successful dag run exists
                 Key: AIRFLOW-1204
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1204
             Project: Apache Airflow
          Issue Type: Bug
    Affects Versions: Airflow 1.8
            Reporter: Bryan Vanderhoof


I have repeatedly run into a problem with the scheduler automatically scheduling DAGs before
their start_date if a previous run of the DAG exists in the database.

For example, if a new DAG has a start_date of 2017-05-01, it will run starting on that date
as expected. However, if there's a existing run for execution_date 2017-04-01, the scheduler
will automatically generate dag runs for 2017-04-02 through 2017-05-01, completely ignoring
the start_date.

This also happens with backfills. Today, I began backfilling data for January 2017 on a DAG
with a start date of 2017-05-01. The backfill began as expected, but as soon as some of the
January tasks began completing, the scheduler also created dag runs for every day in February,
March, and April.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message