reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rogan Carr <rogan.c...@gmail.com>
Subject Re: [Discuss] Harmonized time stamp format
Date Tue, 25 Jul 2017 16:56:17 GMT
Hi All,

The inconsistency in time formats has been something that's bothered me
about parsing REEF logs, and this is something that I'd love to have
standardized.

I've worked with real-time streaming applications in the past, and the
requirements we usually settle on are:

* Needs to be supported across platforms and languages

* Needs to be human readable.
  If you have to copy/paste/parse to determine the time of event, it's next
to worthless for debugging.

* Needs to contain the timezone offset
  Technically, servers can all use UTC, but if we ever want to ask "what
time of the day was it when this event happened", we need the offset.
Especially useful to correlate time-of-day events across different
timezones. (i.e. Maybe there's some administrative event at 3 am local time
in different datacenters that affects performance; this won't be detectable
unless you log the local time.)

* Bonus if it's an international standard format
  Almost guarantees wide library support and recognition, plus people have
already thought through the details.

I propose using this format: "2017-07-25T09:31:22.365-07:00".

This is an implementation of the ISO8601 format, and an abridged version of
.NET's "Round-trip Format" [1], where you only report down to the
millisecond. This timestamp has served me well in the past, and I've never
run into an instance where it's let me down.

Here are common language format strings for these:
const string dotNetIso8601Format = "yyyy-MM-ddTHH:mm:ss.fffzzz";
static final String javaIso8601Format = "yyyy-MM-dd'T'HH:mm:ss.SSSZ";
const javaScriptIso8601Format = 'YYYY-MM-DDTHH:mm:ss.SSSZZ';

Java doesn't have great timestamp support. This format used
SimpleDateFormat and required one correction step to insert a colon in the
timezone offset. I can provide the code if we want to go forward with this.

There is a similar case for Python. I have python format and the necessary
correction code as well.

Best,
Rogan

[1] More about the .NET Round-trip Format:
https://docs.microsoft.com/en-us/dotnet/standard/base-types/how-to-round-trip-date-and-time-values


On Tue, Jul 25, 2017 at 8:44 AM, Saikat Kanjilal <sxk1969@gmail.com> wrote:

> Markus,
> Before rendering feedback on this I wanted to understand the use case some
> more, are these logs only used by the engineers and customers of reef ?
>  Also if that is the case it seems like a simpler version of a time stamp
> would suffice as I think this is mainly used for troubleshooting, correct?
>
> What usually works for me a simple YYYY-MM-DD-HH-MM-SS when
> troubleshooting and debugging issues.
>
> My 2 pennies.
>
>
>
> Sent from my iPhone
>
> > On Jul 25, 2017, at 8:23 AM, Markus Weimer <markus@weimo.de> wrote:
> >
> > Hi,
> >
> > right now, we don't use the same timestamp format in our logs across
> > Java and .NET, and we aren't even consistent within one language
> > environment. I believe we should strive to harmonize that, but am not
> > quite sure what format to settle on.
> >
> > How about this as a strawman: YYYY-MM-DD-HH-SS-MMM, e.g.
> 2017-07-25-08-23-123
> >
> > Would this work? If not, what should be our standard?
> >
> > Thanks,
> >
> > Markus
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message