kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Neha Narkhede (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-955) After a leader change, messages sent with ack=0 are lost
Date Wed, 28 Aug 2013 17:14:51 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13752590#comment-13752590
] 

Neha Narkhede commented on KAFKA-955:
-------------------------------------

This is great. +1. One improvement on logging -

        info(("Send the close connection response due to error handling produce request "
+
          "[clientId = %s, correlationId = %s, topicAndPartition = %s] with Ack=0")
          .format(produceRequest.clientId, produceRequest.correlationId, produceRequest.topicPartitionMessageSizeMap.mkString("[",",","]")))

Here we only want to print the topic and partition, so it seems that we should be printing
the keys of the map, not the entire map ?
produceRequest.topicPartitionMessageSizeMap.keySet.mkString(",")

I can make this change on checkin.
                
> After a leader change, messages sent with ack=0 are lost
> --------------------------------------------------------
>
>                 Key: KAFKA-955
>                 URL: https://issues.apache.org/jira/browse/KAFKA-955
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Jason Rosenberg
>            Assignee: Guozhang Wang
>         Attachments: KAFKA-955.v1.patch, KAFKA-955.v1.patch, KAFKA-955.v2.patch, KAFKA-955.v3.patch,
KAFKA-955.v4.patch, KAFKA-955.v5.patch, KAFKA-955.v6.patch, KAFKA-955.v7.patch
>
>
> If the leader changes for a partition, and a producer is sending messages with ack=0,
then messages will be lost, since the producer has no active way of knowing that the leader
has changed, until it's next metadata refresh update.
> The broker receiving the message, which is no longer the leader, logs a message like
this:
> Produce request with correlation id 7136261 from client  on partition [mytopic,0] failed
due to Leader not local for partition [mytopic,0] on broker 508818741
> This is exacerbated by the controlled shutdown mechanism, which forces an immediate leader
change.
> A possible solution to this would be for a broker which receives a message, for a topic
that it is no longer the leader for (and if the ack level is 0), then the broker could just
silently forward the message over to the current leader.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message