kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From onurkaraman <...@git.apache.org>
Subject [GitHub] kafka pull request #2816: KAFKA-5028: convert kafka controller to a single-t...
Date Thu, 06 Apr 2017 05:37:37 GMT
GitHub user onurkaraman opened a pull request:

    https://github.com/apache/kafka/pull/2816

    KAFKA-5028: convert kafka controller to a single-threaded event queue model

    The goal of this ticket is to improve controller maintainability by simplifying the controller's
concurrency semantics. The controller code has a lot of shared state between several threads
using several concurrency primitives. This makes the code hard to reason about.
    
    This ticket proposes we convert the controller to a single-threaded event queue model.
We add a new controller thread which processes events held in an event queue. Note that this
does not mean we get rid of all threads used by the controller. We merely delegate all work
that interacts with controller local state to this single thread. With only a single thread
accessing and modifying the controller local state, we no longer need to worry about concurrent
access, which means we can get rid of the various concurrency primitives used throughout the
controller.
    
    Performance is expected to match existing behavior since the bulk of the existing controller
work today already happens sequentially in the ZkClient’s single ZkEventThread.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/onurkaraman/kafka KAFKA-5028

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/kafka/pull/2816.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2816
    
----
commit d74cad8874f1a9dd2b781c3033921f562d4e7630
Author: Onur Karaman <okaraman@linkedin.com>
Date:   2017-04-06T05:20:23Z

    KAFKA-5028: convert kafka controller to a single-threaded event queue model
    
    The goal of this ticket is to improve controller maintainability by simplifying the controller's
concurrency semantics. The controller code has a lot of shared state between several threads
using several concurrency primitives. This makes the code hard to reason about.
    
    This ticket proposes we convert the controller to a single-threaded event queue model.
We add a new controller thread which processes events held in an event queue. Note that this
does not mean we get rid of all threads used by the controller. We merely delegate all work
that interacts with controller local state to this single thread. With only a single thread
accessing and modifying the controller local state, we no longer need to worry about concurrent
access, which means we can get rid of the various concurrency primitives used throughout the
controller.
    
    Performance is expected to match existing behavior since the bulk of the existing controller
work today already happens sequentially in the ZkClient’s single ZkEventThread.

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message