kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Russell Sayers (Jira)" <j...@apache.org>
Subject [jira] [Updated] (KAFKA-10685) --to-datetime passed to kafka-consumer-groups interpreting microseconds wrong
Date Thu, 05 Nov 2020 06:01:00 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-10685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Russell Sayers updated KAFKA-10685:
-----------------------------------
    Description: 
f you pass more than 3 decimal places for the fractional seconds of the datetime, the microseconds
get interpreted as milliseconds.

{{kafka-consumer-groups --bootstrap-server kafka:9092 \}}
{{--reset-offsets \}}
{{--group webserver-avro \}}
{{--topic driver-positions-avro \}}
{{--to-datetime "}}{{2020-11-05T00:46:48.002237400}}{{" \}}
{{--dry-run}}

Relevant code [here|https://github.com/apache/kafka/blob/2.7/clients/src/main/java/org/apache/kafka/common/utils/Utils.java#L1304].
The datetime is being turned into Nov 5, 2020 1:24:05.400 because SimpleDateFormat is adding 2237400
milliseconds to Nov 5, 2020 00:46:48.

Experimenting with getDateTime:
 * getDateTime("2020-11-05T00:46:48.000") -> 1604537208000
 * getDateTime("2020-11-05T00:46:48.000+0800") -> 1604508408000 - correct the formatting
string allows for ZZZ timezones
 * getDateTime("2020-11-05T00:46:48.000123") -> 1604537208123 - note this ends with 123
milliseconds.

The pattern string is "yyyy-MM-dd'T'HH:mm:ss.SSS".  So SimpleDateFormat interprets "000123"
as 123 milliseconds. See the stackoverflow answer [here|https://stackoverflow.com/a/21235602/109102].

The fix?  Remove any digits after more than 3 characters after the decimal point, or raise
an exception. The code would still need to allow the RFC822 timezone, i.e Sign TwoDigitHours
Minutes.

 

  was:
If you pass more than 3 decimal places for the fractional seconds of the datetime, the microseconds
get interpreted as milliseconds.

{{kafka-consumer-groups --bootstrap-server kafka:9092 \}}
{{ --reset-offsets \}}
{{ --group webserver-avro \}}
{{ --topic driver-positions-avro \}}
{{ --to-datetime "2020-11-05T00:46:48.002237400" \}}
{{ --dry-run}}

Relevant code [here|https://github.com/apache/kafka/blob/2.7/clients/src/main/java/org/apache/kafka/common/utils/Utils.java#L1304].

Experimenting with getDateTime:
 * getDateTime("2020-11-05T00:46:48.000") -> 1604537208000
 * getDateTime("2020-11-05T00:46:48.000+0800") -> 1604508408000 - correct the formatting
string allows for ZZZ timezones
 * getDateTime("2020-11-05T00:46:48.000123") -> 1604537208123 - note this ends with 123
milliseconds.

The pattern string is "yyyy-MM-dd'T'HH:mm:ss.SSS".  So SimpleDateFormat interprets "000123"
as 123 milliseconds. See the stackoverflow answer [here|https://stackoverflow.com/a/21235602/109102].

The fix?  Remove any digits after more than 3 characters after the decimal point, or raise
an exception. The code would still need to allow the RFC822 timezone, i.e Sign TwoDigitHours
Minutes.

 


> --to-datetime passed to kafka-consumer-groups interpreting microseconds wrong
> -----------------------------------------------------------------------------
>
>                 Key: KAFKA-10685
>                 URL: https://issues.apache.org/jira/browse/KAFKA-10685
>             Project: Kafka
>          Issue Type: Bug
>    Affects Versions: 2.7.0
>            Reporter: Russell Sayers
>            Priority: Minor
>
> f you pass more than 3 decimal places for the fractional seconds of the datetime, the
microseconds get interpreted as milliseconds.
> {{kafka-consumer-groups --bootstrap-server kafka:9092 \}}
> {{--reset-offsets \}}
> {{--group webserver-avro \}}
> {{--topic driver-positions-avro \}}
> {{--to-datetime "}}{{2020-11-05T00:46:48.002237400}}{{" \}}
> {{--dry-run}}
> Relevant code [here|https://github.com/apache/kafka/blob/2.7/clients/src/main/java/org/apache/kafka/common/utils/Utils.java#L1304].
The datetime is being turned into Nov 5, 2020 1:24:05.400 because SimpleDateFormat is adding 2237400
milliseconds to Nov 5, 2020 00:46:48.
> Experimenting with getDateTime:
>  * getDateTime("2020-11-05T00:46:48.000") -> 1604537208000
>  * getDateTime("2020-11-05T00:46:48.000+0800") -> 1604508408000 - correct the formatting
string allows for ZZZ timezones
>  * getDateTime("2020-11-05T00:46:48.000123") -> 1604537208123 - note this ends with
123 milliseconds.
> The pattern string is "yyyy-MM-dd'T'HH:mm:ss.SSS".  So SimpleDateFormat interprets
"000123" as 123 milliseconds. See the stackoverflow answer [here|https://stackoverflow.com/a/21235602/109102].
> The fix?  Remove any digits after more than 3 characters after the decimal point, or
raise an exception. The code would still need to allow the RFC822 timezone, i.e Sign TwoDigitHours
Minutes.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message