flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-8354) Flink Kafka connector ignores Kafka message headers
Date Thu, 13 Sep 2018 06:01:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-8354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16613054#comment-16613054
] 

ASF GitHub Bot commented on FLINK-8354:
---------------------------------------

tzulitai edited a comment on issue #6615: [FLINK-8354] [flink-connectors] Add ability to access
and provider Kafka headers
URL: https://github.com/apache/flink/pull/6615#issuecomment-420892843
 
 
   @alexeyt820 I think it is not easily possible to resolve the conflicts between this PR
and #6105, which also touches the `KeyedDeserializationSchema` and follows a somewhat different
approach.
   
   I'm also not found of wrapping the Kafka record bytes into `Record`; that basically always
ties deserialization of the bytes with access to other meta information, and makes it hard
to reuse some already existing deserialization formats such as `AvroDeserializationSchema`.
   
   Is it ok if I open a new PR that is based on this one and #6105, and see what I come up
with? I might not be able to do that this week, but I can do that as soon as I finish with
what I'm currently busy with.
   
   cc @FredTing would that also be ok for you?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Flink Kafka connector ignores Kafka message  headers 
> -----------------------------------------------------
>
>                 Key: FLINK-8354
>                 URL: https://issues.apache.org/jira/browse/FLINK-8354
>             Project: Flink
>          Issue Type: Bug
>          Components: Kafka Connector
>         Environment: Kafka 0.11.0.0
> Flink 1.4.0
> flink-connector-kafka-0.11_2.11 
>            Reporter: Mohammad Abareghi
>            Assignee: Aegeaner
>            Priority: Major
>              Labels: pull-request-available
>
> Kafka has introduced notion of Header for messages in version 0.11.0.0  https://issues.apache.org/jira/browse/KAFKA-4208.
> But flink-connector-kafka-0.11_2.11 which supports kafka 0.11.0.0 ignores headers when
consuming kafka messages. 
> It would be useful in some scenarios, such as distributed log tracing, to support message
headers to FlinkKafkaConsumer011 and FlinkKafkaProducer011. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message