flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fabian Hueske <fhue...@gmail.com>
Subject Re: StreamSQL procTime granularity
Date Thu, 06 Apr 2017 11:17:00 GMT
Hi Stefano,

thanks for starting this discussion. I think treating processing time with
a nanosecond resolution might be difficult for a few reasons:
- We would need to treat it in all operators as nanotime (joins, etc.)
- Flink does not provide tooling for nanotime resolution. The internal
timer service is on milliseconds.
- We could not use the testing facilities that the DataStream API offers
because they work on milliseconds.

I'm not sure what the benefits of switching to nanoseconds would be.
Sure, higher resolution sounds good, but on the other hand processing time
is somewhat arbitrary anyway.

Best, Fabian

2017-04-04 10:23 GMT+02:00 Stefano Bortoli <stefano.bortoli@huawei.com>:

> Hi guys,
>
> Based on the discussion about time management in
> https://github.com/apache/flink/pull/3641 , does it make sense to use
> nanoTime for procTime semantic aggregation processing? This way we will not
> have the possibility of events falling in the same "millisecond" processing
> bucket and keep a consistent aggregation sorting (also in the state). I
> understand that event-time cannot be managed in nanosecond as java does not
> give wall-clock time in nanoseconds, but for the procTime within the JVM we
> should be safe.
>
> Please let me know what you think.
>
> Best,
> Stefano
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message