airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maxime Beauchemin <maximebeauche...@gmail.com>
Subject Re: Cron schedule with DST-aware timezone
Date Mon, 13 May 2019 21:39:19 GMT
It would be great if people can provide failing unit tests as PR with clear
expectations stated out as code. It makes it easier for people to get
consensus on expectations and for anyone to jump in and implement a fix.

Max

On Mon, May 13, 2019 at 12:48 PM David Klosowski <davidk@thinknear.com>
wrote:

> Damian is correct.  We've observed that exact behavior and noticed the
> timedelta logic is dubiously broken for DST but works for CRON.
>
> On Mon, May 13, 2019 at 12:38 PM Shaw, Damian P. <
> damian.shaw.2@credit-suisse.com> wrote:
>
> > Hi,
> >
> > I’m not part of the Airflow team but I came to the same conclusion, that
> > the behavior is the opposite of what the documentation specifies. When
> > using the cron notation then DST is properly honored, when using
> timedelta
> > it is not. I played around with the DAG.following_schedule method to
> > satisfy this was the case.
> >
> > I’ve had production instance of airflow based on this that successfully
> > respected the March/April DST changes in many timezones.
> >
> > Regards
> > Damian
> >
> > From: Jiahao Chen [mailto:jhchen@google.com.INVALID]
> > Sent: Monday, May 13, 2019 2:08 PM
> > To: dev@airflow.apache.org
> > Subject: Cron schedule with DST-aware timezone
> >
> > Hi team,
> >
> > I have a question about the expected behavior of Airflow scheduler when
> > the schedule_interval is a cron expression and the start_date is in a
> > timezone with DST.
> >
> > Based on the Airflow documentation
> > https://airflow.apache.org/timezone.html#cron-schedules, the DST change
> > will be ignored if schedule_interval is a cron expresion (e.g. '0 17 * *
> > *'). And it gives an example that the GMT offset will not change
> regardless
> > how DST changes. If I'm understanding it correctly, that means if I
> upload
> > a DAG with a schedule_interval of "0 17 * * *" and a start_date of
> > 2019-03-15 17:00 PST(GMT-8) which is before the DST change on March 10,
> the
> > Airflow scheduler will always start the DAG on 5 pm everyday GMT-8 even
> > after the DST change on March 10.
> >
> > However, that is not the behavior I've seen with my experimental code
> (see
> > attachments). It looks like the the Airflow is actually taking the DST
> into
> > account, since the execution time is always 17:00 locally, which is 1
> hour
> > off on the GMT after the DST change.
> >
> > Could you please confirm the behavior of Airflow scheduler in this use
> > case?
> >
> > Thank you!
> > Jiahao
> >
> >
> >
> >
> >
> ===============================================================================
> >
> > Please access the attached hyperlink for an important electronic
> > communications disclaimer:
> > http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html
> >
> ===============================================================================
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message