kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Khushal Kumthekar (JIRA)" <j...@apache.org>
Subject [jira] [Created] (KAFKA-8540) Improve InsertField connect transform to allow inserting processed timestamp
Date Fri, 14 Jun 2019 17:28:00 GMT
Khushal Kumthekar created KAFKA-8540:
----------------------------------------

             Summary: Improve InsertField connect transform to allow inserting processed timestamp
                 Key: KAFKA-8540
                 URL: https://issues.apache.org/jira/browse/KAFKA-8540
             Project: Kafka
          Issue Type: Improvement
          Components: KafkaConnect
    Affects Versions: 2.2.1
            Reporter: Khushal Kumthekar


We have requirement to insert a new field that gets populated with the timestamp when kafka
message was processed by the connector.

There is no such transform currently available. *"InputField"* transform provides *timestamp.field* option
but that sets the field value to the timestamp when messaged landed in kafka and not when
message was processed by the connector.

Here is the proposal

Enhance *InputField* transform to add a new option called *"processedTimestamp.field"* that
will allow users to insert new field with value of message processed timestamp as shown in
below configuration
{code:java}
"transforms": "InsertField",
"transforms.InsertField.type":"org.apache.kafka.connect.transforms.InsertField$Value",
"transforms.InsertField.processedTimestamp.field": "field_name"{code}
here is the potential code block to set this value
{code:java}
updatedValue.put(processedTimestampField.name, new Date()); {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message