Mailing-List: contact dev-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@flink.apache.org
MIME-Version: 1.0
In-Reply-To: <633F19C6003CFF44891E456BB8269E55014E6B92@lhreml505-mbs.china.huawei.com>
References: <633F19C6003CFF44891E456BB8269E55014E6B92@lhreml505-mbs.china.huawei.com>
From: Fabian Hueske <fhueske@gmail.com>
Date: Thu, 6 Apr 2017 13:17:00 +0200
Message-ID: <CAAdrtT0KDrK0cfK9_ySvZCWKJgzsP5WY5HdzHhYivAdPCheC5Q@mail.gmail.com>
Subject: Re: StreamSQL procTime granularity
To: "dev@flink.apache.org" <dev@flink.apache.org>
Content-Type: multipart/alternative; boundary=94eb2c18db3aab014c054c7da762
archived-at: Thu, 06 Apr 2017 11:17:35 -0000

--94eb2c18db3aab014c054c7da762
Content-Type: text/plain; charset=UTF-8

Hi Stefano,

thanks for starting this discussion. I think treating processing time with
a nanosecond resolution might be difficult for a few reasons:
- We would need to treat it in all operators as nanotime (joins, etc.)
- Flink does not provide tooling for nanotime resolution. The internal
timer service is on milliseconds.
- We could not use the testing facilities that the DataStream API offers
because they work on milliseconds.

I'm not sure what the benefits of switching to nanoseconds would be.
Sure, higher resolution sounds good, but on the other hand processing time
is somewhat arbitrary anyway.

Best, Fabian

2017-04-04 10:23 GMT+02:00 Stefano Bortoli <stefano.bortoli@huawei.com>:

> Hi guys,
>
> Based on the discussion about time management in
> https://github.com/apache/flink/pull/3641 , does it make sense to use
> nanoTime for procTime semantic aggregation processing? This way we will not
> have the possibility of events falling in the same "millisecond" processing
> bucket and keep a consistent aggregation sorting (also in the state). I
> understand that event-time cannot be managed in nanosecond as java does not
> give wall-clock time in nanoseconds, but for the procTime within the JVM we
> should be safe.
>
> Please let me know what you think.
>
> Best,
> Stefano
>
>

--94eb2c18db3aab014c054c7da762--