kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-5501) use async zookeeper apis everywhere
Date Sat, 24 Jun 2017 09:51:02 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-5501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16061888#comment-16061888
] 

ASF GitHub Bot commented on KAFKA-5501:
---------------------------------------

GitHub user onurkaraman opened a pull request:

    https://github.com/apache/kafka/pull/3427

    KAFKA-5501 [WIP]: use async zookeeper apis everywhere

    Synchronous zookeeper writes means that we wait an entire round trip before doing the
next write. With respect to the controller, these synchronous writes are happening at a per-partition
granularity in several places, so partition-heavy clusters suffer from the controller doing
many sequential round trips to zookeeper.
    - PartitionStateMachine.electLeaderForPartition updates leaderAndIsr in zookeeper on transition
to OnlinePartition. This gets triggered per-partition sequentially with synchronous writes
during controlled shutdown of the shutting down broker's replicas for which it is the leader.
    - ReplicaStateMachine updates leaderAndIsr in zookeeper on transition to OfflineReplica
when calling KafkaController.removeReplicaFromIsr. This gets triggered per-partition sequentially
with synchronous writes for failed or controlled shutdown brokers.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/onurkaraman/kafka KAFKA-5501

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/kafka/pull/3427.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #3427
    
----
commit d118a542c4e08ea1e14fc577f64c746946ac3dea
Author: Onur Karaman <okaraman@linkedin.com>
Date:   2017-06-01T20:14:34Z

    use async zookeeper apis everywhere
    
    Synchronous zookeeper writes means that we wait an entire round trip before doing the
next write. With respect to the controller, these synchronous writes are happening at a per-partition
granularity in several places, so partition-heavy clusters suffer from the controller doing
many sequential round trips to zookeeper.
    - PartitionStateMachine.electLeaderForPartition updates leaderAndIsr in zookeeper on transition
to OnlinePartition. This gets triggered per-partition sequentially with synchronous writes
during controlled shutdown of the shutting down broker's replicas for which it is the leader.
    - ReplicaStateMachine updates leaderAndIsr in zookeeper on transition to OfflineReplica
when calling KafkaController.removeReplicaFromIsr. This gets triggered per-partition sequentially
with synchronous writes for failed or controlled shutdown brokers.

----


> use async zookeeper apis everywhere
> -----------------------------------
>
>                 Key: KAFKA-5501
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5501
>             Project: Kafka
>          Issue Type: Sub-task
>            Reporter: Onur Karaman
>            Assignee: Onur Karaman
>
> Synchronous zookeeper writes means that we wait an entire round trip before doing the
next write. These synchronous writes are happening at a per-partition granularity in several
places, so partition-heavy clusters suffer from the controller doing many sequential round
trips to zookeeper.
> * PartitionStateMachine.electLeaderForPartition updates leaderAndIsr in zookeeper on
transition to OnlinePartition. This gets triggered per-partition sequentially with synchronous
writes during controlled shutdown of the shutting down broker's replicas for which it is the
leader.
> * ReplicaStateMachine updates leaderAndIsr in zookeeper on transition to OfflineReplica
when calling KafkaController.removeReplicaFromIsr. This gets triggered per-partition sequentially
with synchronous writes for failed or controlled shutdown brokers.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message