airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "sulphide (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AIRFLOW-2250) start_date logic seems broken in 1.9
Date Fri, 23 Mar 2018 20:21:00 GMT

     [ https://issues.apache.org/jira/browse/AIRFLOW-2250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

sulphide updated AIRFLOW-2250:
------------------------------
    Description: 
after upgrading to 1.9 some of our dags get execution_date's that are not aligned with our
hardcoded start_date. this breaks our sensors, e.g.

 

[2018-03-23 20:06:21,781] \{base_task_runner.py:98} INFO - Subtask: [2018-03-23 20:06:21,780]
\{sensors.py:243} INFO - Poking for upstream_2017-12-06_2000.upstream_task on 2018-03-22T19:51:52.050448 

 
in this case our ExternalTaskSensor has a start_date with a start_time of 2100 and is pointing
at the upstream dag and task with an execution_delta of 1 hour. the problem is that the task_instance
gets an execution_date of 2018-03-022T19:51:52.050448 but should be getting an execution_date
of 2018-03-22T21:00:00

  was:
after upgrading to 1.9 some of our dags get start times that are not aligned with our hardcoded
start_date. this breaks our sensors, e.g.

 

[2018-03-23 20:06:21,781] \{base_task_runner.py:98} INFO - Subtask: [2018-03-23 20:06:21,780]
\{sensors.py:243} INFO - Poking for upstream_2017-12-06_2000.upstream_task on 2018-03-22T19:51:52.050448 

 
in this case our ExternalTaskSensor has a start_date with a start_time of 2100 and is pointing
at the upstream dag and task with an execution_delta of 1 hour.


> start_date logic seems broken in 1.9
> ------------------------------------
>
>                 Key: AIRFLOW-2250
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-2250
>             Project: Apache Airflow
>          Issue Type: Bug
>            Reporter: sulphide
>            Priority: Major
>
> after upgrading to 1.9 some of our dags get execution_date's that are not aligned with
our hardcoded start_date. this breaks our sensors, e.g.
>  
> [2018-03-23 20:06:21,781] \{base_task_runner.py:98} INFO - Subtask: [2018-03-23 20:06:21,780]
\{sensors.py:243} INFO - Poking for upstream_2017-12-06_2000.upstream_task on 2018-03-22T19:51:52.050448 
>  
> in this case our ExternalTaskSensor has a start_date with a start_time of 2100 and is
pointing at the upstream dag and task with an execution_delta of 1 hour. the problem is that
the task_instance gets an execution_date of 2018-03-022T19:51:52.050448 but should be getting
an execution_date of 2018-03-22T21:00:00



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message