flume-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Roshan Naik (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLUME-2781) A Kafka Channel defined as parseAsFlumeEvent=false cannot be correctly used by a Flume source
Date Thu, 31 Mar 2016 21:09:25 GMT

    [ https://issues.apache.org/jira/browse/FLUME-2781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15220676#comment-15220676
] 

Roshan Naik commented on FLUME-2781:
------------------------------------

Thanks for confirming  [~gherreros]

I got a confused when i saw this stmt  in some of the automated comments here in this jira.

{quote}"Kafka Channel with parseAsFlumeEvent=true should write data as is, not as flume events."
.{quote}

which i believe is exactly the opposite of what is intended.

> A Kafka Channel defined as parseAsFlumeEvent=false cannot be correctly used by a Flume
source
> ---------------------------------------------------------------------------------------------
>
>                 Key: FLUME-2781
>                 URL: https://issues.apache.org/jira/browse/FLUME-2781
>             Project: Flume
>          Issue Type: Improvement
>    Affects Versions: v1.6.0
>            Reporter: Gonzalo Herreros
>            Assignee: Gonzalo Herreros
>              Labels: easyfix, patch
>             Fix For: v1.7.0
>
>         Attachments: FLUME-2781.patch
>
>
> When a Kafka channel is configured as parseAsFlumeEvent=false, the channel will read
events from the topic as text instead of serialized Avro Flume events.
> This is useful so Flume can read from an existing Kafka topic, where other Kafka clients
publish as text.
> However, if you use a Flume source on that channel, it will still write the events as
Avro so it will create an inconsistency and those events will fail to be read correctly.
> Also, this would allow a Flume source to write to a Kafka channel and any Kafka subscriber
to listen to Flume events passing through without binary dependencies.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message