flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shuyi Chen <suez1...@gmail.com>
Subject Timestamp(timezone) conversion bug in non blink Table/SQL runtime
Date Mon, 22 Jul 2019 18:46:26 GMT
Hi all,

Currently, in the non-blink table/SQL runtime, Flink used
SqlFunctions.internalToTimestamp(long v) from Calcite to convert event time
(in long) to java.sql.Timestamp. However, as discussed in the recent
Calcite mailing list (Jul. 19, 2019), SqlFunctions.internalToTimestamp()
assumes the input timestamp value is in the current JVM’s default timezone
(which is unusual), NOT milliseconds since epoch. And
SqlFunctions.internalToTimestamp() is used to convert timestamp value in
the current JVM’s default timezone to milliseconds since epoch, which
java.sql.Timestamp constructor takes. Therefore, the results will not only
be wrong, but change if the job runs in machines on different timezones as
well. (The only exception is that all your production machines uses UTC
timezone.)

Here is an example, if the user input value is 0 (00:00:00 UTC on 1 January
1970), and the table/SQL runtime runs in a machine in PST (UTC-8), the
output sql.Timestamp after SqlFunctions.internalToTimestamp() will become
28800000 millisec since epoch (08:00:00 UTC on 1 January 1970); And with
the same input, if the table/SQL runtime runs again in a different machine
in EST (UTC-5), the output sql.Timestamp after
SqlFunctions.internalToTimestamp() will become 18000000 millisec since
epoch (05:00:00 UTC on 1 January 1970).

More details are captured in
https://issues.apache.org/jira/browse/FLINK-13372. Please let me know your
thoughts and correct me if I am wrong. Thanks a lot.

Shuyi

Mime
View raw message