beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xu Mingmin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-1514) change default timestamp in KafkaIO
Date Mon, 20 Feb 2017 20:07:45 GMT

    [ https://issues.apache.org/jira/browse/BEAM-1514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15875015#comment-15875015
] 

Xu Mingmin commented on BEAM-1514:
----------------------------------

[~davor], will refer to the naming standard in Cloud PubsubIO.

the lines impacted would be here https://github.com/XuMingmin/beam/blob/6aca4d5238165ead825ec6c55202cebc091e900d/sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaIO.java#L962-L963


curTimestamp = (source.spec.getTimestampFn() == null)
    ? Instant.now() : source.spec.getTimestampFn().apply(record);

With kafka client 0.10, ConsumerRecord.timestamp is there, instead of using Instant.now().

> change default timestamp in KafkaIO
> -----------------------------------
>
>                 Key: BEAM-1514
>                 URL: https://issues.apache.org/jira/browse/BEAM-1514
>             Project: Beam
>          Issue Type: Improvement
>          Components: sdk-java-extensions
>            Reporter: Xu Mingmin
>            Assignee: Xu Mingmin
>
> When user use Kafka 0.10, the field 'timestamp' from Kafka should be used as the default
event timestamp.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message