kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Olson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-1028) per topic configuration of preference for consistency over availability
Date Tue, 04 Mar 2014 01:12:21 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-1028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13918871#comment-13918871
] 

Andrew Olson commented on KAFKA-1028:
-------------------------------------

Patch has been updated with the suggested changes.

Note that I made one additional change which was found to be necessary after rebasing against
the latest trunk code (detected by integration test failures). It appears that the recent
KAFKA-1235 changes result in a Leader of -1 (none) and empty ISR being saved for all partitions
after all brokers in the cluster have been shutdown using a controlled shutdown. This essentially
forces an unclean leader election to occur when a broker is subsequently restarted.  If we
have the configuration set to disallow unclean elections, then we have permanently blocked
our ability to restore a leader.

To address this, I've added logic that prevents the ISR from being updated to an empty set
for any topics that do not allow unclean election. The last surviving member of the ISR will
be preserved in Zookeeper in the event that a final, lone leader broker goes offline -- allowing
this previous leader to resume its leader role when it comes back online. The partition leader
is still updated to -1, which is not problematic.

Please review lines 975-986 of KafkaController and let me know if this change makes sense.

> per topic configuration of preference for consistency over availability
> -----------------------------------------------------------------------
>
>                 Key: KAFKA-1028
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1028
>             Project: Kafka
>          Issue Type: Improvement
>            Reporter: Scott Clasen
>            Assignee: Neha Narkhede
>         Attachments: KAFKA-1028.patch, KAFKA-1028_2014-01-30_13:45:30.patch, KAFKA-1028_2014-03-03_18:48:43.patch
>
>
> As discussed with Neha on the ML.
> It should be possible to configure a topic to disallow unclean leader election, thus
preventing the situation where committed messages can be discarded once a failed leader comes
back online in a situation where it was the only ISR.
> This would open kafka to additional usecases where the possibility of committted messages
being discarded is unacceptable.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message