flume-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Li Ye (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (FLUME-2789) Kafka sink sends messages to the source topic by mistake
Date Mon, 25 Apr 2016 13:41:13 GMT

     [ https://issues.apache.org/jira/browse/FLUME-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Li Ye updated FLUME-2789:
-------------------------
    Attachment: FLUME-2789-2.patch

> Kafka sink sends messages to the source topic by mistake 
> ---------------------------------------------------------
>
>                 Key: FLUME-2789
>                 URL: https://issues.apache.org/jira/browse/FLUME-2789
>             Project: Flume
>          Issue Type: Bug
>          Components: Sinks+Sources
>    Affects Versions: v1.6.0
>         Environment: Flume 1.6
>            Reporter: Alex Tian
>         Attachments: FLUME-2789-1.patch, FLUME-2789-2.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> In my scenario, I need to send messages from a kafka source to a kafka sink , in other
workds, transfering messages from a topic A to another topic B.
>     As a result, I find my kafka sink always sends messages back to topic A.
> The reason in the codes is: 
> KafkaSink.java:
> ****************************************************************************************
> 107         if ((eventTopic = headers.get(TOPIC_HDR)) == null) {
>  
> 108           eventTopic = topic;
>  
> 109         }
> ****************************************************************************************
> If the source is a Kafka source, eventTopic won't be null because it is the source topic
from the msg header. Therefore, the kafka sink will send the message to its source topic by
mistake.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message