kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-3812) State store locking is incorrect
Date Sun, 17 Jul 2016 16:32:20 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-3812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15381412#comment-15381412

ASF GitHub Bot commented on KAFKA-3812:

GitHub user dguy opened a pull request:


    KAFKA-3812 State store locking is incorrect

    Move all state directory creation/locking/unlocking/cleaning to a single class. Don't
release the channel until the lock is released. Refactor code to make use of new class

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/dguy/kafka kafka-3812

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1628
commit b4547162d5d07cd471c1ccf9e84ed57c947bf2e4
Author: Damian Guy <damian.guy@gmail.com>
Date:   2016-07-17T16:29:20Z

    move locking/cleaning/creating/unlocking of task directories into a single class. refactor
code to make use of it


> State store locking is incorrect
> --------------------------------
>                 Key: KAFKA-3812
>                 URL: https://issues.apache.org/jira/browse/KAFKA-3812
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>    Affects Versions:
>            Reporter: Eno Thereska
>            Assignee: Damian Guy
>             Fix For:
> In {{ProcessorStateManager}} in the {{lockStateDirectory}} method, we close a channel
if the lock is null. However, as described in https://docs.oracle.com/javase/7/docs/api/java/nio/channels/FileLock.html,
"...closing a channel releases all locks held by the Java virtual machine on the underlying
file regardless of whether the locks were acquired via that channel or via another channel
open on the same file. It is strongly recommended that, within a program, a unique channel
be used to acquire all locks on any given file.". Thus closing this channel sometimes leads
to the other lock in {{ProcessorStateManager}}, called {{directoryLock}} to be released. 
> Problem is reproduced as part of the steps in KAFKA-3805, when two processes are using
the same state directory.

This message was sent by Atlassian JIRA

View raw message