airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vineet Goel (JIRA)" <j...@apache.org>
Subject [jira] [Created] (AIRFLOW-433) Allow tasks to be scheduled in the current schedule interval instead of previous
Date Tue, 16 Aug 2016 21:43:20 GMT
Vineet Goel created AIRFLOW-433:
-----------------------------------

             Summary: Allow tasks to be scheduled in the current schedule interval instead
of previous
                 Key: AIRFLOW-433
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-433
             Project: Apache Airflow
          Issue Type: New Feature
            Reporter: Vineet Goel


I understand that airflow chose to schedule dag runs for the previous schedule interval because
of the way data pipelines for batch processing are usually designed. For example, currently
a daily job is scheduled for a dag run with execution date of Monday on a Tuesday. 

However this is a bit weird for jobs that need to provide the same day of running as the execution_date
(or {{ ds }}). I can get around this by using the date_add macro function for daily jobs but
this workaround fails in the case of jobs that are scheduled to run on weekdays and should
provide the same day in the context.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message