airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Heuermann (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-1056) Single dag run triggered when un-pausing job with catchup=False
Date Thu, 30 Mar 2017 07:06:41 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15948525#comment-15948525
] 

Andrew Heuermann commented on AIRFLOW-1056:
-------------------------------------------

Another idea could be to also update the start_date when un-pausing a dag with catchup=False.

> Single dag run triggered when un-pausing job with catchup=False
> ---------------------------------------------------------------
>
>                 Key: AIRFLOW-1056
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1056
>             Project: Apache Airflow
>          Issue Type: Bug
>    Affects Versions: 1.8.0
>            Reporter: Andrew Heuermann
>
> When "catchup=False" a single job run is still triggered when un-pausing a dag when there
are missed run windows. 
> It updates the dag.start_date here to prevent the backfill: https://github.com/apache/incubator-airflow/blob/bb39078a35cf2bceea58d7831d7a2028c8ef849f/airflow/jobs.py#L770.
> But it looks like the function schedules dags based on a window (using sequential run
times as lower and upper bounds) it still schedules one dag run.
> The only ideas I have now on how to fix is to pass in the time when the scheduler last
ran and use that as the lower bound of the window, but not sure how easy that is to get to.
There might be a simpler solution I'm missing.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message