kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dong Lin (JIRA)" <j...@apache.org>
Subject [jira] [Created] (KAFKA-5879) Controller should read the latest IsrChangeNotification znodes when handling IsrChangeNotification event
Date Wed, 13 Sep 2017 00:28:00 GMT
Dong Lin created KAFKA-5879:
-------------------------------

             Summary: Controller should read the latest IsrChangeNotification znodes when
handling IsrChangeNotification event
                 Key: KAFKA-5879
                 URL: https://issues.apache.org/jira/browse/KAFKA-5879
             Project: Kafka
          Issue Type: Bug
            Reporter: Dong Lin
            Assignee: Dong Lin
            Priority: Critical


Currently controller can be very inefficient in handling IsrChangeNotification event because
it may need to access znode O(n^2) times to handle O(n) IsrChangeNotification znodes.

For example, say there are 100 IsrChangeNotification nodes added to the zookeeper. This will
generate 100 IsrChangeNotification events with children [1], [1, 2], [1, 2, 3], ... [1, 2,
.. 100]. Let's say the controller now needs to handle the IsrChangeNotification event with
children [1, 2, ... 100]. Controller will read zookeeper 100 times, delete these 100 znodes,
which further generates 100 IsrChangeNotification events with children count from 0 to 99.

The main cause of the problem is that, controller will attempt to access zookeeper n times,
where n is the count of the children at the time the IsrChangeNotification event is generated,
even though there is no IsrChangeNotification znodes in the zookeeper.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message