kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jun Rao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-574) KafkaController unnecessarily reads leaderAndIsr info from ZK
Date Thu, 01 Nov 2012 16:39:12 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13488810#comment-13488810

Jun Rao commented on KAFKA-574:

Thanks for patch v2. Just one more comment:

20. ReplicaSatemachine.handleStateChange(): In the OfflineReplica state, after the isr is
updated, we need to update the leaderAndIsr cache in controller context. Also, is leaderAndIsrIsEmpty
better than leaderAndIsrOpt?

Could you run the basic system tests and make sure that they pass?

<kafka_home>/system_test/ $ python –u –B system_test_runner.py 2>&1 | tee
system_test_output_`date +%s`.log
> KafkaController unnecessarily reads leaderAndIsr info from ZK
> -------------------------------------------------------------
>                 Key: KAFKA-574
>                 URL: https://issues.apache.org/jira/browse/KAFKA-574
>             Project: Kafka
>          Issue Type: Bug
>          Components: core
>    Affects Versions: 0.8
>            Reporter: Jun Rao
>            Assignee: Prashanth Menon
>            Priority: Blocker
>              Labels: bugs
>         Attachments: KAFKA-574-v1.patch, KAFKA-574-v2.patch
>   Original Estimate: 48h
>  Remaining Estimate: 48h
> KafkaController calls updateLeaderAndIsrCache() in onBrokerFailure(). This is unnecessary
since in onBrokerFailure(), we will make leader and isr change anyway so there is no need
to first read that information from ZK. Latency is critical in onBrokerFailure() since it
determines how quickly a leader can be made online.
> Similarly, updateLeaderAndIsrCache() is called in onBrokerStartup() unnecessarily. In
this case, the controller does not change the leader or the isr. It just needs to send the
current leader and the isr info to the newly started broker. We already cache leader in the
controller. Isr in theory could change any time by the leader. So, reading from ZK doesn't
guarantee that we can get the latest isr anyway. Instead, we just need to get the isr last
selected by the controller (which can be cached together with the leader in the controller).
If the leader epoc in a broker is at or larger than the epoc in the leaderAndIsr request,
the broker can just ignore it. Otherwise, the leader and the isr selected by the controller
should be used. 

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message