airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <>
Subject [jira] [Commented] (AIRFLOW-372) DAGs can run before start_date time
Date Sun, 02 Sep 2018 18:02:03 GMT


Apache Spark commented on AIRFLOW-372:

User 'AndreiDev' has created a pull request for this issue:

> DAGs can run before start_date time
> -----------------------------------
>                 Key: AIRFLOW-372
>                 URL:
>             Project: Apache Airflow
>          Issue Type: Bug
>    Affects Versions: Airflow
>            Reporter: Isaac Steele
>            Priority: Major
> If you turn off a DAG in the UI, there seemingly is no way to prevent "missed" runs to
schedule after the DAG is turned back on. I thought the workaround for this, since it is not
a parameterized option to prevent, would be to update the start_date in the DAG code before
turning the DAG back on. This does not work, and therefore the scheduler is running dag_runs
*before* the listed start_date.
> To reproduce:
> # Create a DAG with a schedule_interval
> # Let the DAG run at least once
> # Turn off the DAG in the UI
> # Allow the schedule_interval to pass at least twice
> # Update the start_date in the DAG to be be after the two interval time
> # (I then removed the compiled python file and restarted airflow/scheduler just to make
> # Turn DAG back on in UI
> Result: All dag_runs that were "missed" while the DAG was turned off run, despite the
start_date being later.
> Ideally the start_date would always be honored. And also there would be a parameter to
just not run any "missed" dag_runs.

This message was sent by Atlassian JIRA

View raw message