airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From harish singh <harish.sing...@gmail.com>
Subject Re: Adjusting DAG Schedules (For Daylight Savings Time, And In General)
Date Sun, 12 Mar 2017 05:52:12 GMT
Right now our airflow UI shows the time 06:51 UTC  while the correct time
is 05:51 UTC


- Harish

On Thu, Mar 9, 2017 at 1:53 PM, Rob Goretsky <robert.goretsky@gmail.com>
wrote:

> I realized after sending this that some of this behavior about the updated
> 'start_date' not taking effect is explained / addressed in the "Proposal to
> simplify start/end dates" thread on this mailing list.  Seems like
> basically 'start_date' updates are effectively ignored, but
> 'scheduler_interval' updates are taken as a delta from the last
> execution_date.  Still would be curious to hear insight from anyone who has
> had to deal with this!
>
> Thanks,
> Rob
>
> On Thu, Mar 9, 2017 at 4:47 PM, Rob Goretsky <robert.goretsky@gmail.com>
> wrote:
>
> > With Daylight Savings Time upon us, I was wondering if anyone has had to
> > address this issue -- While I understand that right now Airflow is not
> > timezone-aware, and runs all of its jobs in GMT/UTC time, my team
> delivers
> > reports to stakeholders that want to consistently see all data reported
> > through Midnight **Eastern Time**.
> >
> > Right now we have a DAG is scheduled to run at 05:00 GMT, which
> correlates
> > to Midnight Eastern time.   After this weekend, we'll need the DAG
> > scheduled to run at 04:00GMT instead, so that it still correlates to
> > Midnight eastern.   If we just try to modify the DAG Python definition to
> > change the 'start_date', this doesn't seem to take effect - that is, the
> > scheduler continues running the DAG at 05:00GMT. So, a few questions:
> >
> > (1) Once a DAG has been running, why don't changes to the Python
> > 'start_date' seem to take effect?  It seems we always need to create a
> > different dag with a different dag_id.   Is this something about the way
> > the history is stored in the database, and is it something we could
> > possibly tweak in the database directly if we wanted to?
> >
> > (2) Has anyone else dealt with this issue of needing to adjust a large
> set
> > of DAGs for DST?  Or am I the only unlucky ones whose stakeholders don't
> > speak GMT?
> >
> > Thanks for all of the help!
> >
> > -rob
> >
> >
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message