airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "sulphide (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-2250) execution_date logic seems broken in 1.9
Date Wed, 09 May 2018 14:30:00 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-2250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16468913#comment-16468913
] 

sulphide commented on AIRFLOW-2250:
-----------------------------------

we've noticed that even if we 'fix' a dag with the above workaround if we have a dag with
catchup=False and more than a day has passed since the dag was turned on then the dag will
create the broken execution dates when we turn it on. so this bug is related to the dag's
ability to create its first execution date.

> execution_date logic seems broken in 1.9
> ----------------------------------------
>
>                 Key: AIRFLOW-2250
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-2250
>             Project: Apache Airflow
>          Issue Type: Bug
>            Reporter: sulphide
>            Priority: Major
>         Attachments: Screen Shot 2018-03-23 at 4.47.03 PM.png
>
>
> after upgrading to 1.9 some of our dags get execution_date's that are not aligned with
our hardcoded start_date. this breaks our sensors, e.g.
>  
> [2018-03-23 20:06:21,781] \{base_task_runner.py:98} INFO - Subtask: [2018-03-23 20:06:21,780]
\{sensors.py:243} INFO - Poking for upstream_2017-12-06_2000.upstream_task on 2018-03-22T19:51:52.050448 
>  
> in this case our ExternalTaskSensor has a start_date with a start_time of 2100 and is
pointing at the upstream dag and task with an execution_delta of 1 hour. 
> the problem: the task_instance gets an execution_date of 2018-03-022T19:51:52.050448
but should be getting an execution_date of 2018-03-22T21:00:00
> it seems that if we manually set the execution date on the first dag run then every subsequent
dag run has the right execution date (so far).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message