kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brian Hawkins (JIRA)" <j...@apache.org>
Subject [jira] [Created] (KAFKA-5847) Plugin option to filter consumer and producer messages on the broker
Date Wed, 06 Sep 2017 20:42:00 GMT
Brian Hawkins created KAFKA-5847:
------------------------------------

             Summary: Plugin option to filter consumer and producer messages on the broker
                 Key: KAFKA-5847
                 URL: https://issues.apache.org/jira/browse/KAFKA-5847
             Project: Kafka
          Issue Type: Wish
          Components: core
    Affects Versions: 0.10.2.1
            Reporter: Brian Hawkins


The idea is that I could specify a plugin that would receive a message, after authorization
but before it is written to the log.  The plugin could then modify or reject the message before
passing it on.  A good place for this would be in KafkaApis.scala in handleProducerRequest.

Similarly a message could be modified before it is sent to the consumer.

I have two use cases in mind: 
1. deal with large messages, the interceptor/filter would write the message to a large storage
server (think s3).
2. encrypt data before being written to the log.

I'm planning on doing this work, just curious if others are interested so I can make a pull
request of it.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message