airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rob Goretsky <robert.goret...@gmail.com>
Subject Adjusting DAG Schedules (For Daylight Savings Time, And In General)
Date Thu, 09 Mar 2017 21:47:14 GMT
With Daylight Savings Time upon us, I was wondering if anyone has had to
address this issue -- While I understand that right now Airflow is not
timezone-aware, and runs all of its jobs in GMT/UTC time, my team delivers
reports to stakeholders that want to consistently see all data reported
through Midnight **Eastern Time**.

Right now we have a DAG is scheduled to run at 05:00 GMT, which correlates
to Midnight Eastern time.   After this weekend, we'll need the DAG
scheduled to run at 04:00GMT instead, so that it still correlates to
Midnight eastern.   If we just try to modify the DAG Python definition to
change the 'start_date', this doesn't seem to take effect - that is, the
scheduler continues running the DAG at 05:00GMT. So, a few questions:

(1) Once a DAG has been running, why don't changes to the Python
'start_date' seem to take effect?  It seems we always need to create a
different dag with a different dag_id.   Is this something about the way
the history is stored in the database, and is it something we could
possibly tweak in the database directly if we wanted to?

(2) Has anyone else dealt with this issue of needing to adjust a large set
of DAGs for DST?  Or am I the only unlucky ones whose stakeholders don't
speak GMT?

Thanks for all of the help!

-rob

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message