airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maxime Beauchemin <maximebeauche...@gmail.com>
Subject Re: UTC everywhere requirement
Date Tue, 07 Jun 2016 19:15:02 GMT
I have little visibility on the requirements of the community around making
Airflow timezone aware as I have an internal UTC clock in my brain. For the
record some of the engineers here answer in UTC when casually asking "what
time is it?" ):

I'd be interested in reading a proposal / impact analysis on the subject
though.

Max

On Tue, Jun 7, 2016 at 5:26 AM, Eric Johnson <johnson.eric@gmail.com> wrote:

> Where is the flaw in this thinking?
>
> So if airflow consistently uses naive datetime objects, won't they all end
> up using the same timezone and thus be equally right in the same kind of
> wrong way? I would only anticipate a problem when there is a mismatch when
> comparing a naive datetime object with a timezone aware datetime object.
> But that doesn't seem to be the case here.
>
> I'm not trying to ignore the wisdom offered from others about embracing UTC
> everywhere especially given the pains daylight saving time. It's just that
> this requirement can be a bit of an imposition for some of us who are
> trying to embrace airflow in an environment where its not UTC everywhere.
> So my line of questioning is aimed at trying to suss out just what breaks
> when you use another timezone.
>
> -Eric
>
> On Mon, Jun 6, 2016 at 7:10 PM, George Leslie-Waksman <
> george@cloverhealth.com> wrote:
>
> > There are three common ways of handling timezones: 1) use timezone aware
> > datetime objects, 2) use timezone naive datetime objects and make sure
> any
> > system running your software has its clock set toUTC, or 3) be very, very
> > sad because have potentially inconsistent times in your logs, scheduling,
> > etc.
> >
> > Airflow uses timezone naive datetime objects everywhere so you're stuck
> > with option 2 or option 3.
> >
> > Historically, handling timezones has been a pain so most people try to go
> > with option 2. I would argue that it's gotten easy enough in Python that
> it
> > would be worth moving airflow to timezone aware datetime objects and
> making
> > everything a little bit easier/safer from a DevOps perspective.
> >
> > On Mon, Jun 6, 2016 at 2:50 PM Lance Norskog <lance.norskog@gmail.com>
> > wrote:
> >
> > > Any larger use of log management will end up with servers in two
> > different
> > > time zones. The ops team will inevitably set all machines to run with
> > time
> > > zone UTC. After that, all of the logs will be in UTC.
> > >
> > >
> > >
> > > On Mon, Jun 6, 2016 at 9:49 AM, Edwards, Jesse <Jesse.Edwards@nike.com
> >
> > > wrote:
> > >
> > > > While I can't speak to Airflow's exact requirement.
> > > >
> > > >
> > > > For me the really clear reason is:
> > > >
> > > >
> > > > 1. All other timezones are based off UTC +/- some amount. Thus
> > > calculating
> > > > the time in any given timezone is trivial.
> > > >
> > > > 2. UTC does not have daylight savings! It really sucks and throws
> > systems
> > > > out of whack when they think there are two 2AM hours on Time change
> =).
> > > >
> > > >
> > > > Jesse
> > > >
> > > > ________________________________
> > > > From: Eric Johnson <johnson.eric@gmail.com>
> > > > Sent: Monday, June 6, 2016 9:28:54 AM
> > > > To: dev@airflow.incubator.apache.org
> > > > Subject: UTC everywhere requirement
> > > >
> > > > Hi -
> > > >
> > > > I see that at the moment Airflow wants everything to be in UTC.
> > > >
> > > > Can those who understand this requirement speak a bit about where in
> > the
> > > > code this dependency exists? I took a cursory glance through the code
> > in
> > > a
> > > > brief attempt to understand the scope of this issue and it wasn't
> > > > immediately obvious where this was even assumed. I don't doubt it.
> Just
> > > > didn't spot it.
> > > >
> > > > I can certainly understand the need for all of airflow to run in the
> > same
> > > > time zone. But I find it odd that it really needs everything to be
> UTC
> > > and
> > > > only UTC.
> > > >
> > > > Eric
> > > >
> > >
> > >
> > >
> > > --
> > > Lance Norskog
> > > lance.norskog@gmail.com
> > > Redwood City, CA
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message