kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-5787) StoreChangeLogReader needs to restore partitions that were added post initialization
Date Fri, 25 Aug 2017 09:39:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-5787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16141424#comment-16141424
] 

ASF GitHub Bot commented on KAFKA-5787:
---------------------------------------

GitHub user dguy opened a pull request:

    https://github.com/apache/kafka/pull/3736

    KAFKA-5787: StoreChangelogReader needs to restore partitions that were added post initialization

    If a task fails during initialization due to a LockException, its changelog partitions
are not immediately added to the StoreChangelogReader as the thread doesn't hold the lock.
However StoreChangelogReader#restore will be called and it sets the initialized flag. On a
subsequent successfull call to initialize the new tasks the partitions are added to the StoreChangelogReader,
however as it is already initialized these new partitions will never be restored. So the task
would remain in a non-running state forever.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/dguy/kafka kafka-5787

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/kafka/pull/3736.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #3736
    
----
commit 451b6a746cc5173b22d8110497bbbe33842d18a3
Author: Damian Guy <damian.guy@gmail.com>
Date:   2017-08-25T09:36:05Z

    StoreChangelogReader needs to restore partitions that were added post initialization

----


> StoreChangeLogReader needs to restore partitions that were added post initialization
> ------------------------------------------------------------------------------------
>
>                 Key: KAFKA-5787
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5787
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>    Affects Versions: 0.11.0.1, 1.0.0
>            Reporter: Damian Guy
>            Assignee: Damian Guy
>            Priority: Blocker
>
> Investigation of {{KStreamRepartitionJoinTest}} failures uncovered this bug. If a task
fails during initialization due to a {{LockException}}, its changelog partitions are not immediately
added to the {{StoreChangelogReader}} as the thread doesn't hold the lock. However {{StoreChangelogReader#restore}}
will be called and it sets the initialized flag. On a subsequent successfull call to initialize
the new tasks the partitions are added to the {{StoreChangelogReader}}, however as it is already
initialized these new partitions will never be restored. So the task will remain in a non-running
state forever



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message