kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-6126) Reduce rebalance time by not checking if created topics are available
Date Wed, 13 Dec 2017 21:23:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-6126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289929#comment-16289929

ASF GitHub Bot commented on KAFKA-6126:

GitHub user mjsax opened a pull request:


    KAFKA-6126: Remove unnecessary topics created check

    ### Committer Checklist (excluded from commit message)
    - [ ] Verify design and implementation 
    - [ ] Verify test coverage and CI build status
    - [ ] Verify documentation (including upgrade notes)

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/mjsax/kafka kafka-6126-remove-topic-check-on-rebalance-2

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #4322
commit c0a8b4868cf979e47a25d5e837b26e62e2aac212
Author: Matthias J. Sax <matthias@confluent.io>
Date:   2017-12-13T21:06:18Z

    KAFKA-6126: Remove unnecessary topics created check


> Reduce rebalance time by not checking if created topics are available
> ---------------------------------------------------------------------
>                 Key: KAFKA-6126
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6126
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>    Affects Versions: 1.0.0
>            Reporter: Matthias J. Sax
>            Assignee: Matthias J. Sax
>             Fix For: 1.1.0
> Within {{StreamPartitionAssignor#assign}} we create new topics and afterwards wait in
an "infinite loop" until topic metadata propagated throughout the cluster. We do this, to
make sure topics are available when we start processing.
> However, with this approach we "extend" the time in the rebalance phase and thus are
not responsive (no calls to `poll` for liveness check and {{KafkaStreams#close}} suffers).
Thus, we might want to remove this check and handle potential "topic not found" exceptions
in the main thread gracefully.

This message was sent by Atlassian JIRA

View raw message