kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantine Karantasis (JIRA)" <j...@apache.org>
Subject [jira] [Created] (KAFKA-5567) With transformations that mutate the topic-partition committing offsets should to refer to the original topic-partition
Date Fri, 07 Jul 2017 00:56:00 GMT
Konstantine Karantasis created KAFKA-5567:
---------------------------------------------

             Summary: With transformations that mutate the topic-partition committing offsets
should to refer to the original topic-partition
                 Key: KAFKA-5567
                 URL: https://issues.apache.org/jira/browse/KAFKA-5567
             Project: Kafka
          Issue Type: Bug
          Components: KafkaConnect
    Affects Versions: 0.11.0.0, 0.10.2.1, 0.10.2.0
            Reporter: Konstantine Karantasis
            Assignee: Konstantine Karantasis
             Fix For: 0.11.0.1



  When a chain of transformations (SMTs) that mutate a record's topic-partition is applied,
then Connect is unable to map the transformed record to its original topic-partition. This
affects committing offsets. 

 Currently, in order to reproduce the issue one could use the {{TimestampRouter}} transformation
with a sink connector such as the {{FileStreamSinkConnector}}.

  In this ticket we'll address the issue for connectors that don't manage/commit their offsets
themselves. For the connectors that do such management, broader API changes are required to
supply the connectors with the necessary information that will allow them to map a transformed
record to the original. 




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message