kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Neha Narkhede (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-955) After a leader change, messages sent with ack=0 are lost
Date Wed, 28 Aug 2013 17:14:51 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13752590#comment-13752590

Neha Narkhede commented on KAFKA-955:

This is great. +1. One improvement on logging -

        info(("Send the close connection response due to error handling produce request "
          "[clientId = %s, correlationId = %s, topicAndPartition = %s] with Ack=0")
          .format(produceRequest.clientId, produceRequest.correlationId, produceRequest.topicPartitionMessageSizeMap.mkString("[",",","]")))

Here we only want to print the topic and partition, so it seems that we should be printing
the keys of the map, not the entire map ?

I can make this change on checkin.
> After a leader change, messages sent with ack=0 are lost
> --------------------------------------------------------
>                 Key: KAFKA-955
>                 URL: https://issues.apache.org/jira/browse/KAFKA-955
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Jason Rosenberg
>            Assignee: Guozhang Wang
>         Attachments: KAFKA-955.v1.patch, KAFKA-955.v1.patch, KAFKA-955.v2.patch, KAFKA-955.v3.patch,
KAFKA-955.v4.patch, KAFKA-955.v5.patch, KAFKA-955.v6.patch, KAFKA-955.v7.patch
> If the leader changes for a partition, and a producer is sending messages with ack=0,
then messages will be lost, since the producer has no active way of knowing that the leader
has changed, until it's next metadata refresh update.
> The broker receiving the message, which is no longer the leader, logs a message like
> Produce request with correlation id 7136261 from client  on partition [mytopic,0] failed
due to Leader not local for partition [mytopic,0] on broker 508818741
> This is exacerbated by the controlled shutdown mechanism, which forces an immediate leader
> A possible solution to this would be for a broker which receives a message, for a topic
that it is no longer the leader for (and if the ack level is 0), then the broker could just
silently forward the message over to the current leader.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message