kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yingji Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Created] (KAFKA-5803) KafkaProducer should ignore invalid record from ProducerInterceptors
Date Mon, 28 Aug 2017 23:09:00 GMT
Yingji Zhang created KAFKA-5803:
-----------------------------------

             Summary: KafkaProducer should ignore invalid record from ProducerInterceptors
                 Key: KAFKA-5803
                 URL: https://issues.apache.org/jira/browse/KAFKA-5803
             Project: Kafka
          Issue Type: Bug
          Components: clients
    Affects Versions: 0.11.0.0, 0.10.2.1, 0.10.2.0, 0.10.1.1
            Reporter: Yingji Zhang


Right now KafkaProducer does not verify the input record emitted from ProducerInterceptors.
Thus if client sends a message to a non exist topic partition or ProducerInterceptors has
a bug, or decides to early exit by returning a null, the doSend method of KafkaProducer will
hung on metadata refreshment until maxBlockTimeMs has elapsed. This means production traffic
will be blocked for each producer send call.

In general, I feel like we should have a circuit break mechanism to allow early exit from
producer. To do that, we can either (1) allow ProducerInterceptors to throw some type of CircuitBreaker
exception or (2) let ProducerInterceptors return null as the protocol to indicate a disaster
situation and make KafkaProducer's doSend early exit on null input.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message