airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rob Goretsky <robert.goret...@gmail.com>
Subject Re: Adjusting DAG Schedules (For Daylight Savings Time, And In General)
Date Thu, 09 Mar 2017 21:53:38 GMT
I realized after sending this that some of this behavior about the updated
'start_date' not taking effect is explained / addressed in the "Proposal to
simplify start/end dates" thread on this mailing list.  Seems like
basically 'start_date' updates are effectively ignored, but
'scheduler_interval' updates are taken as a delta from the last
execution_date.  Still would be curious to hear insight from anyone who has
had to deal with this!

Thanks,
Rob

On Thu, Mar 9, 2017 at 4:47 PM, Rob Goretsky <robert.goretsky@gmail.com>
wrote:

> With Daylight Savings Time upon us, I was wondering if anyone has had to
> address this issue -- While I understand that right now Airflow is not
> timezone-aware, and runs all of its jobs in GMT/UTC time, my team delivers
> reports to stakeholders that want to consistently see all data reported
> through Midnight **Eastern Time**.
>
> Right now we have a DAG is scheduled to run at 05:00 GMT, which correlates
> to Midnight Eastern time.   After this weekend, we'll need the DAG
> scheduled to run at 04:00GMT instead, so that it still correlates to
> Midnight eastern.   If we just try to modify the DAG Python definition to
> change the 'start_date', this doesn't seem to take effect - that is, the
> scheduler continues running the DAG at 05:00GMT. So, a few questions:
>
> (1) Once a DAG has been running, why don't changes to the Python
> 'start_date' seem to take effect?  It seems we always need to create a
> different dag with a different dag_id.   Is this something about the way
> the history is stored in the database, and is it something we could
> possibly tweak in the database directly if we wanted to?
>
> (2) Has anyone else dealt with this issue of needing to adjust a large set
> of DAGs for DST?  Or am I the only unlucky ones whose stakeholders don't
> speak GMT?
>
> Thanks for all of the help!
>
> -rob
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message