kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "cuiyang (JIRA)" <j...@apache.org>
Subject [jira] [Created] (KAFKA-4951) KafkaProducer may writes duplicated message sometimes
Date Mon, 27 Mar 2017 05:55:41 GMT
cuiyang created KAFKA-4951:

             Summary: KafkaProducer may writes duplicated message sometimes
                 Key: KAFKA-4951
                 URL: https://issues.apache.org/jira/browse/KAFKA-4951
             Project: Kafka
          Issue Type: Bug
    Affects Versions:
            Reporter: cuiyang

I foud that KafkaProducer may writes duplicated message sometimes, which is happend when:
     In Sender thread:
         -> this.selector.poll()    //write the message, such as "abc";
         -> handleTimedOutRequests(responses,updatedNow);  //Judge whether the message
 "abc" which is sent above is expired or timeout,  and the judge is  based on the parameter
 this.requestTimeoutMs and updatedNow;  
         -> response.request().callback().onComplete()
             -> completeBatch(batch,Errors.NETWORK_EXCEPTION,-1L,correlationId,now);  
//If themessage was judged as expired, then it will be reenqueued and send repeatly next loop;
                 -> this.accumulator.reenqueue(batch,now);
The problem comes out:  If the message "abc" is sent successfully to broker, but it may be
judged to expired, so the message will be sent repeately next loop, which make the message

I can reproduce this scenario normally.
In my opinion, I think Send::handleTimedOutRequests() is not much useful, because the response
of sending request from broker is succesfully and has no error. And this function  will induce
to the duplicated message problems.

This message was sent by Atlassian JIRA

View raw message